Table of Contents
Generative AI Systems
What Makes ChatGPT and Other Robust Generative AI Systems Different from Each Other and How Do They Operate?
A cursory glance at the news headlines suggests that generative artificial intelligence is a buzzword of the day. Some of those headlines, like OpenAI’s ChatGPT, a chatbot with the remarkable ability to produce text that looks to have been written by a human, may have been composed using generative AI.
However, when someone refers to “generative AI,” what exactly do they mean?
When people discussed artificial intelligence (AI) a few years ago, prior to the general AI boom, they typically meant machine learning models that could predict outcomes from data. These models could be trained with millions of examples, for instance, to forecast the signs of a particular X-ray tumor or the chance that a given user will miss payments on a loan.
Generative AI can be conceptualized as a kind of machine learning model that is trained to produce new data in addition to predictions about a given dataset. A generative AI system is one that picks up the ability to create more objects that resemble the training data.
Differentiating between the core principles of machine learning in generic artificial intelligence and other types of AI can be challenging. . According to Philip Isola, an associate professor of electrical engineering and computer science at MIT and a member of the Computer Science and Artificial Intelligence Lab (CSAIL), “the same algorithm can often be used for both.”
Although ChatGPT and its equivalents have generated a lot of excitement, the technology is not totally novel. These potent machine learning models are the result of over 50 years of research and computational advancement.
Growing Intricacy:
A very basic model called a Markov Chain serves as an early illustration of generative AI. The statistical method used to model the behavior of random processes was first developed in 1906 by the Russian mathematician Andrey Markov, after whom this technique was named. Markov models have long been used in machine learning for tasks like next-word prediction, like in autocomplete email programs.
A Markov model uses text prediction to determine the next word in a sentence by looking up the word before it or a few words before it. While these basic models can predict to some extent, Thomas Sibel, Professor of Electrical Engineering and Computer Science at MIT, notes that they are not very good at producing coherent text—a point that Jakola, a member of CSAIL and IDSAIL (Institute for Data, Systems, and Society) also makes.
“Before the last ten years, we have been producing things similar to this for decades. However, the primary distinction in this case is the intricacy of the objects we can produce and the scope at which we can train these models,” the speaker states.
A few years ago, the goal of research was to identify the optimal machine learning algorithm for a particular dataset. But now that the emphasis has shifted, a lot of researchers are using massive datasets—possibly with hundreds of thousands or even millions of data points—to train models that yield remarkable results.
ChatGPT and related systems function using a Markov model-like underlying model. But there’s a big difference: ChatGPT is far bigger and more sophisticated, having been trained on a ton of data (in this case, publicly accessible text on the internet).
Words and sentences occur in somewhat dependent sequences throughout this extensive corpus of writing. By predicting what might come next, this repetition helps the model comprehend how the text can be divided into statistical segments, which facilitates the creation of new text.
Greater Architecture Power:
Large datasets have aided in the development of generic AI, but significant advances in research have also led to the development of increasingly sophisticated deep-learning architectures.
Generative Adversarial Networks (GAN) are a machine learning architecture that was first presented by researchers at the University of Montreal in 2014. GANs use a pair of models that collaborate: one learns to produce an output (such as an image), and the other learns to distinguish between the generator’s output and actual data. The generator learns to produce more realistic outputs by trying to trick the discriminator. This kind of model forms the basis of the image generator StyleGAN.
A year later, Stanford University and University of California, Berkeley researchers introduced a Progressive Model that iteratively improves the output. These models have been applied to produce realistic-looking images by learning to generate new data samples that resemble those in the training dataset. Steganography, the text-to-image generation system at the heart of stacked diffusion, is one example of such an application.
The Transformer architecture, which was unveiled by Google researchers in 2017, has been crucial in the development of massive language models such as ChatGPT. A Transformer is used in natural language processing to encode each word in a text sequence into a token. It then builds an attention map by identifying relationships between every other token. When the Transformer is creating new text, this attention map aids in its understanding of context.
These are but a few applications for generative artificial intelligence.
An assortment of uses:
The conversion of input into a set of tokens—numerical representations of data chunks—is what ties together all of these viewpoints. Essentially, you can use these techniques to create new data that appears similar as long as your original data can be transformed into this common token format.
It’s really getting close to how a standard CPU can take any kind of data and start synthesizing it in a unified manner, says Isola. Your mileage may vary; it depends on how much data you have and how difficult it is to extract signals.
This gives generative AI a wide range of new applications. To teach a computer vision model to recognize objects, for example, Isola’s group is using generative AI to create synthetic image data that can be used to train other intelligent systems.
In the meantime, Jakola’s team is creating legitimate crystal structures that specify brand-new protein structures or innovative materials using generative AI. He clarified that a generative model can learn the relationships that make structures stable and formable if crystal structures are displayed, much like it can learn the dependence of language.
However, even though generative models can produce amazing outcomes, they might not be the best choice for all kinds of data. Professor Devavrat Shah, an IDSS member and electrical engineering and computer science professor at MIT, claims that generic AI models perform better than traditional machine learning techniques for tasks involving making predictions on structured data, such as tabular data in a spreadsheet.
“In my opinion, their greatest contribution lies in developing an excellent and user-friendly interface for machines. In the past, in order to communicate with machines, humans needed to learn their language. As of right now, this interface can converse with both computers and people,” adds Shah.
Raising the alarm:
In order to ask questions of real customers, call centers are now using generative AI chatbots. However, this application draws attention to a potential risk associated with the use of these models: worker displacement.
Furthermore, hate speech and incorrect statements may be amplified by generative AI, which has the ability to introduce biases from the training set. The models can commit literary theft by creating content that seems to have been written by a particular human author, which could lead to copyright violations.
Shah, on the other hand, suggests that generic AI can empower creative artists who might not otherwise have the means to produce, by enabling them to use a generic tool to assist in generating creative content.
He sees generic AI revolutionizing economics across a range of domains in the future. Isola believes that generative AI has a bright future in the construction industry, where it may be able to produce a chair’s blueprint rather than just an image of one.
More intelligent AI agents are what he generally sees as the future application of generic AI systems. “”I think there are some similarities and differences between how these models work and what we know about how the human brain works. Generative AI, in my opinion, is one of the technologies that will enable agents to think creatively and imaginatively, to generate intriguing ideas or plans,” states Isola.
