What the “G” in ChatGPT Means

The “G”, “P” and “T” in ChatGPT stand for “Generative”, “Pre-Trained” and “Transformer” respectively. The true power comes from the “T” or “Transformer”. But, for me, the “G” or “Generative” is the more foundational idea.

In this text, I try to clarify what “Generative” means within the context of Artificial Intelligence and Machine Learning, as I’d to a 9-year-old.

[Obviously, “Generative” has a formal and mathematical meaning — as does its counterpart “Discriminative”. I won’t go into these here, but if you’re interested, Wikipedia has a decent explanation.]

Let’s suppose you might be a 9-year-old.

Suppose we play the next two guessing games.

• Game 1 “Guess the tune”: I play a brief tune on the piano, and also you guess what it’s. For instance, I play the tune of “Mary had slightly lamb”, and also you guess it.
• Game 2 “Play the tune”: I name a tune, and also you play it on the piano. For instance, I say, “Mary had slightly lamb” and also you play it.

Now, suppose you might be superb at Game 1 (“Guess the tune”). Does that naturally imply that you just are also good at Game 2 (“Play the tune”)?

No. There are numerous individuals who shall be superb at guessing the names of tunes, but can’t play them on a piano, and even hum or whistle them.

The converse can also be true. You might be good at playing any tune that is known as but may not have the option to call a tune that’s played to you.

Many Machine Learning models are like people good at Game 1 or Game 2, but not each.

For instance, you possibly can train a model that may guess the name of any tune played (e.g., Shazam), and you can even train a model that may play any tune named (Spotify is just not exactly a Machine Learning model, but that’s what it does).

The flexibility to be good at either Game 1 or else Game 2, is typically known as a ability. In other words, the person or the model can discriminate between tunes or names.

Now, there are people who find themselves good at each Game 1 and Game 2. They will each play a tune named, and name a tune played.

People who find themselves good at each games are sometimes capable of do two things that “Discriminative” people (those that are good at just one game) cannot do.

1. Given a made-up name of a song (say, “Dhara had slightly Frog”) they will make up a completely latest song to match the name.
2. Conversely, in the event you make-up a tune, they will provide you with a plausible name for it.

They will do the above since the appear to have the option to expand on or generalize or on the knowledge of names and tunes that they’ve.

This ability is often known as .

Generative Models, like ChatGPT or DALL.E have this very same ability.

For instance, you possibly can say “Dhara had slightly Frog” to DALL.E and it may provide you with a completely latest picture that matches that title. That may be a principle reason why they’re so powerful and increasingly popular.