If you’ve already used the AI chatbot, you’ve probably been amazed by its ability to quickly generate human-like responses. Naturally, you want to know how does ChatGPT work?
If you want to know how something works, whoever made the thing is usually the best person to ask.
If you want to know how a Tesla car works, you might ask Elon Musk.
In the case of ChatGPT, it was created by a company called OpenAI.
On the OpenAI website, there’s an article with answers to “commonly asked questions about ChatGPT.”
One of the questions … How does ChatGPT work? Bingo!
And the answer, written by Natalie Staudacher (AI Specialist at OpenAI):
“ChatGPT is fine-tuned from GPT-3.5, a language model trained to produce text. ChatGPT was optimized for dialogue by using Reinforcement Learning with Human Feedback (RLHF) – a method that uses human demonstrations and preference comparisons to guide the model toward desired behavior.”
— OpenAI, “What is ChatGPT?”
Unless you have a Ph.D. in Computer Science, that answer probably isn’t very helpful.
In fact, you probably have even more questions now:
- What is a language model?
- What is GPT?
- What does “trained” mean?
- What does “fine-tuned” mean?
- What is RLHF?
Not to worry! Let’s go through those questions one by one.
At the end, we’ll give a simple answer to the question, “How does ChatGPT work?” Without any of the technical jargon.
What is a language model?
A language model is a machine learning model that can conduct a probability distribution over sequences of words.
The model is trained using a large dataset of text.
A simple use case for a language model is to predict the most appropriate word to fill a blank space in a sentence.
GPT is an example of a language model.
What is GPT?
In the answer above, Staudacher mentions GPT-3.5 specifically.
GPT-3.5 is one of the latest models in the GPT series. It’s the model that currently powers the free version of ChatGPT (the paid version, ChatGPT Plus, is powered by GPT-4).
Instead of getting into any more detail on the different models, let’s just focus on plain old GPT.
GPT is an acronym that stands for Generative Pre-trained Transformer.
What does “Generative” mean?
“Generative” means, well, it can generate stuff.
What can it generate?
It can generate text in response to prompts. In other words, you can ask it a question and it can answer.
If you’ve been hearing the buzz phrase “Generative AI,” that’s the same “Generative” we’re talking about here.
What does “trained” mean?
Before we talk about “Pre-trained,” let’s start with what “trained” means.
You can think of GPT as a super smart robot.
One day, the robot went to the biggest library in the world and read all the books.
Now the robot has a ton of knowledge on different topics. The robot has been “trained.”
To train a language model like GPT, you need a dataset (i.e., the books in the library).
The original GPT-1, for example, was trained on two datasets: Common Crawl (billions of web pages) and BookCorpus (11,000 unpublished books).
Now, the data needs to be translated into units called tokens before GPT can read it, but we won’t get into that here.
If you want to learn more about the technical details of training a language model, check out this article in The New York Times (more simplified) and this blog post by Hugging Face (more complex).
What does “Pre-trained” mean?
Okay, now you know what it means for a language model like GPT to be trained, but what does “Pre-trained” mean?
When OpenAI announced GPT-1 on June 11, 2018, this is how they explained the system:
“Our system works in two stages; first we train a transformer model on a very large amount of data in an unsupervised manner—using language modeling as a training signal—then we fine-tune this model on much smaller supervised datasets to help it solve specific tasks.”
— OpenAI, “Improving language understanding with unsupervised learning”
That part where they say “first we train a transformer model on a very large amount of data” is the pre-training.
It’s called pre-training because it’s followed by more training during the fine-tuning phase.
Here is a more technical explanation of pre-training.
What does “fine-tuned” mean?
As we explained above, fine-tuning is the training phase that happens after pre-training.
The pre-training phase gives the model a general understanding of the language and then the fine-tuning phase enables the model to handle specific tasks.
What does “Transformer” mean?
Rick Merritt at NVIDIA says, “A transformer model is a neural network that learns context and thus meaning by tracking relationships in sequential data like the words in this sentence.”
If you want to learn more about how transformers work, check out this lecture in Hugging Face’s NLP Course.
What is Reinforcement Learning with Human Feedback (RLHF)?
As Staudacher explains in the original answer above, RLHF is “a method that uses human demonstrations and preference comparisons to guide the model toward desired behavior.”
This one basically explains itself. The language model is learning based on human feedback.
For example, imagine you’re using ChatGPT and it gives you an answer that doesn’t make sense, so you reply, “That doesn’t make sense.” ChatGPT is now going to remember that particular answer didn’t make sense to a human.
You could even take your feedback one step further and tell ChatGPT, “Try saying it this way next time … “ This is the type of human feedback that ChatGPT is using to learn via RLHF.
Read more on the Hugging Face blog about RLHF.
How does ChatGPT actually work?
Now that we’ve deciphered all the technical jargon, we can give a simple explanation in response to the question, “How does ChatGPT work?”
The basic functionality of ChatGPT is that you ask it questions and it gives you answers.
It’s able to come up with answers because it has learned in much the same way that humans learn, by consuming information and storing it in our brains.
And its answers are human-like because (1) it has learned from texts produced by humans, and (2) it continues to learn based on feedback from humans.
You don’t need to know how ChatGPT works to use it
It’s kinda like driving a car.
If you want to drive down the street, you don’t need to know every technical detail of how an internal-combustion engine converts energy from gas to torque.
But it might help to know a few things, like how to turn on the engine, put the transmission in gear, and slowly apply pressure to the gas pedal.
Knowing some things will make you a better driver. But knowing more than that won’t necessarily be helpful for operating the machine.
It’s the same with ChatGPT and other generative AI tools. You don’t need to know as much as an AI engineer. You just need to know enough to use the thing.
Read more on our blog to learn how to use AI for writing.