Why Zero Trust Information is one of the right paradigms in this age of ChatGPT and Bing chatbot for regular end users?

I thank Carl for sharing You Are Not a Parrot And a chatbot is not a human. And a linguist named Emily M. Bender is very worried what will happen when we forget this by Elizabeth Weil. It is worth reading till the very end several times.

In order to understand the above mentioned article and Zero Trust Information, we may have to refer the lucid overview of Large Language models here by Murray Shanahan first.

What are LLMs (Large Language Models)?

LLMs are generative mathematical models of the statistical distribution of tokens in the vast public corpus of human- generated text, where the tokens in question include words, parts of words, or individual characters including punctuation marks. They are generative because we can sample from them, which means we can ask them questions. But the questions are of the following very specific kind. “Here’s a fragment of text. Tell me how this fragment might go on. According to your model of the statistics of human language, what words are likely to come next?”

Suppose we give an LLM the prompt “The first person to walk on the Moon was ”, and suppose it responds with “Neil Armstrong”. What are we really asking here? In an important sense, we are not really asking who was the first person to walk on the Moon. What we are really asking the model is the following question: Given the statistical distribution of words in the vast public corpus of (English) text, what words are most likely to follow the sequence “The first person to walk on the Moon was ”? A good reply to this question is “Neil Armstrong”. Similarly, we might give an LLM the prompt “Twinkle twinkle ”, to which it will most likely respond “little star”. On one level, for sure, we are asking the model to remind us of the lyrics of a well-known nursery rhyme. But in an important sense what we are really doing is asking it the following question: Given the statistical distribution of words in the public corpus, what words are most likely to follow the sequence “Twinkle twinkle ”? To which an accurate answer is “little star”.

Also a review of Harry G. Frankfurt’s concept of bullshit can be worth. Harry Frankfurt’s idea of bullshit is that it is a form of speech that is not necessarily false, but is not based on truth. It is often used to create a false impression or to avoid engaging with the truth. It is also not intended to convey any meaningful information, but is instead used as a way to create a sense of certainty or to manipulate the listener. The book Calling Bullshit by Carl T. Bergstrom and Jevin D. West may be referred.

Large language model size has been increasing 10x every year for the last few years. This is starting to look like another Moore’s Law. With model sizes approaching thousand billions of parameters, the boundaries between probability and reality are going to get even more blurred. Distinguishing bullshit will be challenging for a layman human mind. Engineering tools based on large language models will be seen as advancing productivity and efficiency in the areas of virtual assistants, image annotation, content creation, cybersecurity etc. As stated here, Pricewaterhouse Cooper’s (PwC) third annual AI Predictions Report has highlighted the importance of focusing on the fundamentals in preparation for large-scale AI projects. Pricewaterhouse Cooper’s (PwC) sees AI as a major game changer. AI could contribute up to $15.7 trillion to the global economy in 2030, more than the current output of China and India combined. Of this, $6.6 trillion is likely to come from increased productivity and $9.1 trillion is likely to come from consumption side effects.

AI and Large Language Models are not going away soon. How do we mitigate the risks? That is a subject of deep research by itself and well elaborated in the article AI might bring huge benefits — if we avoid the risks. The immediate option, for ordinary mortals, like me, is to increase our awareness, further sharpen our senses and increase our mental ability and alertness. I believe this is what brilliant minds like Emily M Bender, Timnit Gebru and others are meaningfully attempting with deepest honesty and sincerity to strike a balance.

Potential harms of large language models can be mitigated by watermarking model output, i.e., embedding signals into generated text that are invisible to humans but algorithmically detectable from a short span of tokens. I am not yet sure about the effectiveness of tools like AI Text Classifier and GPTZero that can distinguish between AI-written and human-written text.

The relevant concept of Zero Trust Information was introduced in AI Homework by Ben Thompson. Ben aptly says that real skill for students will be in verifying the answers the systems, like ChatGPT churns out, i.e. learning how to be a verifier and an editor, instead of a regurgitator. Even before the advent of ChatGPT, many of us are already aware that in today’s world of fake news and information deluge, verifying and editing must be an essential skill for every individual. In his article, Ben suggests that we adhere to the concept of “Zero Trust Information” similar to the paradigm behind Zero Trust Networking.

In You Are Not a Parrot And a chatbot is not a human. And a linguist named Emily M. Bender is very worried what will happen when we forget this Elizabeth Weil says that tech-makers assuming their reality accurately represents the world create many different kinds of problems. The training data for ChatGPT is believed to include most or all of Wikipedia, pages linked from Reddit, a billion words grabbed off the internet. (It can’t include, say, e-book copies of everything in the Stanford library, as books are protected by copyright law.) The humans who wrote all those words online overrepresent white people. They overrepresent men. They overrepresent wealth. What’s more, we all know what’s out there on the internet: vast swamps of racism, sexism, homophobia, Islamophobia, neo-Nazism.

With such background scenarios of biasedness, Ben’s suggestion of “Zero Trust Information” can become one of the necessary tools for survival for common end users.

It is time to re-train our own minds again and again. I repeat Emily M Bender’s rallying cries below.

“Please do not conflate word form and meaning. Mind your own credulity.“

It is high time the difference between form and meaning was well understood. The next three paragraphs are quoted from the blog by Scott Aaronson.

Form is the physical structure of something, while meaning is the interpretation or concept that is attached to that form. For example, the form of a chair is its physical structure – four legs, a seat, and a back. The meaning of a chair is that it is something you can sit on.

This distinction is important when considering whether or not an AI system can be trained to learn semantic meaning. AI systems are capable of learning and understanding the form of data, but they are not able to attach meaning to that data. In other words, AI systems can learn to identify patterns, but they cannot understand the concepts behind those patterns.

For example, an AI system might be able to learn that a certain type of data is typically associated with the concept of “chair.” However, the AI system would not be able to understand what a chair is or why it is used. In this way, we can see that an AI system trained on form can never learn semantic meaning.

#AI #OpenAI #ChatGPT #ZeroTrustInformation #LargeLanguageModels #LLM

cc: @srijit @srijit

Zero Trust Information

Shel Israels’s one line advice, regarding usage of GPT AI, reinforces my thoughts regarding Zero Trust Information as the right paradigm in this age of ChatGPT, Google Bard and Bing chatbot for regular end users.

My advice is to treat GPT AI the same way you treat a blinking yellow light on a dark street: proceed with caution.

#AI #OpenAI #ChatGPT #ZeroTrustInformation #LargeLanguageModels #LLM #GPTAI

cc: @srijit