Generative AI 101
Look at any news source these days and you'll likely see references to generative artificial intelligence (AI) systems like ChatGPT, DALL-E and Google Bard, among others. You may know a bit about what these revolutionary AI models can do and the benefits they offer, but what about the technology behind them? Read on to gain a better understanding of generative AI, including what it is, how it works, as well as some popular use cases.
What is generative AI?
In basic terms, generative AI is a type of AI that can be used to produce new content such as text, images, audio, video, code or synthetic data. These models fall under the category of machine learning (ML), a branch of AI in which algorithms help computers learn and change in response to new data without the help of a human being. Previously, these were largely predictive models that identified and classified patterns in content. With the introduction of generative AI, however, these models are able to produce new outputs based on the data they've been trained on.
As is the case with any successful machine learning model, generative AI relies on the quantity, quality and diversity of the data used to train it. In fact, these models train on massive amounts of data. For example, according to McKinsey & Company, it's estimated that the large language model ChatGPT was trained on 45 terabytes of text data — almost the equivalent of all of the content on the public web. Such a vast amount of training data enables ChatGPT to understand the patterns and structures of natural language. As a result, it learns language nuances and considers the context of the conversation between itself and the user.
The essential guide to AI training data
Discover best practices for the sourcing, labeling and analyzing of training data from TELUS International, a leading provider of AI data solutions.
A brief history of generative AI
The first chatbot, introduced in 1966, is considered to be a precursor to generative AI because it was able to produce short pre-programmed responses to user inquiries. Generative AI as we know it today was built on deep learning, which traces back to the 1950s. Innovations in the field of deep learning remained somewhat quiet until the 1980s and 90s with the introduction of convolutional neural networks (CNNs), commonly applied to analyze images, and recurrent neural networks (RNNs), commonly applied to analyze text sequences.
A huge leap in the development of generative AI came in 2014 with the introduction of generative adversarial networks (GANs). With this breakthrough, AI models could use GANs to convincingly create authentic images, video and audio. A further advancement came in 2017 with the introduction of transformers, which could analyze longer input sequences and determine the dependencies between those inputs.
These advancements led to generative AI's ability to do revolutionary tasks, such as producing a new image based on an existing one; writing poems, essays and news articles; and even producing sound effects and new music tracks.
How does generative AI work?
Generative AI belongs to a class of machine learning known as large language models (LLM). LLMs use deep learning in natural language processing (NLP) tasks. NLP is the umbrella term for any machine's ability to recognize what is said to it, understand its meaning, determine the appropriate action and respond in a language the user will understand.
In deep learning, artificial neural networks (computing systems inspired by the neural networks that make up the human brain) are trained on very large datasets. Similar to how humans learn from experience, the deep learning algorithm performs a task repeatedly until the optimal outcome is achieved. An example is autonomous vehicles. To operate safely, these vehicles have to react to the changing environment around them. The more data (images, videos and lidar) they are trained on, the more optimally they can react to what's happening on the road.
The introduction of GANs in 2014 propelled the development of generative AI forward tremendously. A type of machine learning algorithm, a GAN works by combining two networks, a generator and a discriminator, that compete against each other to perform a task. Ultimately, this competition produces increasingly better results.
When replicating an image, for example, the generator will produce a synthetic version of the underlying data, which is used to try to convince the discriminator that it's the real version. The discriminator then attempts to distinguish between the data that's authentic and the synthetic data. This task is relatively easy at first since the generator creates random patterns that the discriminator can readily distinguish from real data. However, with each subsequent attempt, the generator produces more representative data each time. Eventually, the generator gets really good at generating synthetic data and is able to convince the discriminator that it's real.
In 2017, generative AI model development leapt forward once again with the introduction of transformers. Prior to transformers, recurrent neural networks (RNNs) were (and still are) used in many NLP tasks. RNNs are designed to interpret sequential input data by analyzing one word at a time in a feed-forward nature. However, they often fall short when interpreting long sequences, which make them less suitable for training larger models such as those used for generative AI. The introduction of transformers brought about a new type of neural network architecture that incorporated self-attention mechanisms. It's these mechanisms that allow the models to understand what each word means when used in context. Self-attention refers to the fact that every element in the input data makes a connection to every other element in order to make more relevant predictions. Essentially, transformers are able to interpret the entire input all at once. As a result, they can process massive amounts of data and are able to train very large models like the ones used for generative AI.
One of the most prominent transformer models today is GPT, which stands for generative pre-trained transformer. Developed by OpenAI, the newly introduced GPT-4 is the fourth generation of this transformer model. GPT was created to produce natural language responses to user input. This groundbreaking AI language model took the world by storm when it was introduced in late 2022, likely owing to the fact that it's so simple to use. After a user request is entered into a text box, the AI model processes it and responds based on the information it has available. For example, if you ask ChatGPT to describe a sunset, it will produce something like the following:
A sunset is a beautiful natural phenomenon that occurs when the sun dips below the horizon, casting a warm and colorful glow across the sky. As the sun descends, the colors of the sky shift and change, from bright oranges and pinks to deeper reds and purples. The clouds in the sky may also catch the light, becoming a beautiful canvas for the changing colors. The atmosphere during a sunset is often peaceful and serene, as the world seems to slow down and bask in the beauty of the moment. It's a truly magical experience that can be enjoyed by anyone lucky enough to witness it.
From the prolific conversational output ChatGPT produces when asked to carry out a simple task, it's clear to see how this AI model is having such a big impact on content generation.
Applications for generative AI
Generative AI has a variety of use cases applicable to different industries, various business functions and our everyday lives. Not only can it scale these efforts, it can also quickly produce content for nearly instantaneous results. Some examples include:
This application is one that we are all likely very familiar with. Chatbots provide information, answer inquiries and perform tasks for their users. Their responses are generated in the form of natural language via a chat window.
Generative AI systems can create synthetic data that's similar to real-world data, which can be used for tasks like training machine learning models. Whereas synthetic data is similar to real-world data, it's not necessarily based on any specific real-world data points.
Art and design
There are a myriad of ways generative AI models can be used for visual applications. These include image generation, in which images are generated based on text input; semantic image to photo translation, in which an image is generated from a semantic sketch; and image-to-image conversion, in which the form or color of an image are transformed while the original elements of the image are preserved.
When listing conversational applications of generative AI, there's no end of examples. Articles, scripts, essays, advertisements — you name it — these models can generate it based on user input.
Sentiment analysis is another text-based application, in which the generative AI model attempts to determine the opinion being conveyed in a writing piece. For example, the opinion may be analyzed as being positive, negative or neutral.
Further, text to speech generation can be used to create realistic audio of speech, and speech to speech conversion can be used to generate voice overs for commercials, for example. Generative AI can even be used to compose unique musical pieces.
Code developers might rejoice to know that generative AI can be used for code generation, code completion (code completion is suggested as you type, which saves time and helps to reduce errors), code review (quality checks of existing code with suggestions on how to optimize or improve it) and fixing bugs in the generated code by analyzing code patterns, identifying problems and suggesting corrections.
Marketing and sales
Generative AI can help with client segmentation by combining and analyzing data from various sources to uncover distinctive customer behavior patterns. This is a highly valuable way to target specific audiences in order to increase sales. It also allows for the creation of personalized emails, web pages, ads, captions, taglines and more that better engage the customer based on who they are.
Generative AI could have a vast impact on healthcare management. According to The Times of India, it could facilitate "faster and more accurate diagnoses based on analysis of structured and unstructured data, recommend more effective treatments, and enable personalized care by applying its innate ability to collect, analyze, and interpret vast amounts of patient data."
In the realm of education, generative AI could be used to create personalized lesson plans tailored specifically to a student's needs based on their skill set and past performance to provide them with the most effective education. It could also be used by educators for course design and organization including creating a syllabus, lesson plans and assessments. Further, it could be used for course content creation by assisting in creating new teaching material like test questions, concept summaries, reading lists and more.
Generative AI's impact
There's no question that generative AI is having a massive impact on our lives and the world we live in. It has the potential to bring about great advancements in many industries. While no one can tell exactly what these models will be able to do in the future, one thing is clear — they will only continue to advance as they are trained on more and more data. It's also clear that, as with any AI, generative AI models are designed to work with people, not in lieu of them.