GPT-4 is an improvement, but temper your expectations.

OpenAI stunned the world when it dropped ChatGPT in late 2022. The brand new generative language model is predicted to totally transform entire industries, including media, education, law, and tech. Briefly, ChatGPT threatens to disrupt nearly the whole lot. And even before we had time to actually envision a post-ChatGPT world, OpenAI dropped GPT-4.

In recent months, the speed with which groundbreaking large language models have been released is astonishing. In case you still don’t understand how ChatGPT differs from GPT-3, let alone GPT-4, I don’t blame you.

In this text, we are going to cover the important thing similarities and differences between ChatGPT and GPT-4, including their training methods, performance and capabilities, and limitations.

ChatGPT and GPT-4 each stand on the shoulders of giants, constructing on previous versions of GPT models while adding improvements to model architecture, employing more sophisticated training methods, and increasing the number of coaching parameters.

Each models are based on the transformer architecture, which uses an encoder to process input sequences and a decoder to generate output sequences. The encoder and decoder are connected by an attention mechanism, which allows the decoder to pay more attention to essentially the most meaningful input sequences.

OpenAI’s GPT-4 Technical Report offers little information on GPT-4’s model architecture and training process, citing the “competitive landscape and the security implications of large-scale models.” What we do know is that ChatGPT and GPT-4 are probably trained in an analogous manner, which is a departure from training methods used for GPT-2 and GPT-3. We all know far more concerning the training methods for ChatGPT than GPT-4, so we’ll start there.

ChatGPT

To begin with, ChatGPT is trained on dialogue datasets, including demonstration data, through which human annotators provide demonstrations of the expected output of a chatbot assistant in response to specific prompts. This data is used to fine-tune GPT3.5 with supervised learning, producing a policy model, which is used to generate multiple responses when fed prompts. Human annotators then rank which of the responses for a given prompt produced the most effective results, which is used to coach a reward model. The reward model is then used to iteratively fine-tune the policy model using reinforcement learning.

To sum it up in a single sentence, ChatGPT is trained using Reinforcement Learning from Human Feedback (RLHF), a way of incorporating human feedback to enhance a language model during training. This enables the model’s output to align to the duty requested by the user, fairly than simply predict the following word in a sentence based on a corpus of generic training data, like GPT-3.

GPT-4

OpenAI has yet to reveal details on the way it trained GPT-4. Their Technical Report doesn’t include “details concerning the architecture (including model size), hardware, training compute, dataset construction, training method, or similar.” What we do know is that GPT-4 is a transformer-style generative multimodal model trained on each publicly available data and licensed third-party data and subsequently fine-tuned using RLHF. Interestingly, OpenAI did share details regarding their upgraded RLHF techniques to make the model responses more accurate and fewer prone to veer outside safety guardrails.

After training a policy model (as with ChatGPT), RLHF is utilized in adversarial training, a process that trains a model on malicious examples intended to deceive the model with a purpose to defend the model against such examples in the longer term. Within the case of GPT-4, human domain experts across several fields rate the responses of the policy model to adversarial prompts. These responses are then used to coach additional reward models that iteratively fine-tune the policy model, leading to a model that’s less likely to present out dangerous, evasive, or inaccurate responses.

Capabilities

When it comes to capabilities, ChatGPT and GPT-4 are more similar than they’re different. Like its predecessor, GPT-4 also interacts in a conversational style that goals to align with the user. As you possibly can see below, the responses between the 2 models for a broad query are very similar.

OpenAI agrees that the excellence between the models will be subtle and claims that “difference comes out when the complexity of the duty reaches a sufficient threshold.” Given the six months of adversarial training the GPT-4 base model underwent in its post-training phase, this might be an accurate characterization.

Unlike ChatGPT, which accepts only text, GPT-4 accepts prompts composed of each images and text, returning textual responses. As of the publishing of this text, unfortunately, the capability for using image inputs isn’t yet available to the general public.

Performance

As referenced earlier, OpenAI reports significant improvement in safety performance for GPT-4, in comparison with GPT-3.5 (from which ChatGPT was fine-tuned). Nevertheless, whether the reduction in responses to requests for disallowed content, reduction in toxic content generation, and improved responses to sensitive topics are as a result of the GPT-4 model itself or the extra adversarial testing is unclear presently.

Moreover, GPT-4 outperforms CPT-3.5 on most academic and skilled exams taken by humans. Notably, GPT-4 scores within the ninetieth percentile on the Uniform Bar Exam in comparison with GPT-3.5, which scores within the tenth percentile. GPT-4 also significantly outperforms its predecessor on traditional language model benchmarks in addition to other SOTA models (although sometimes just barely).

Each ChatGPT and GPT-4 have significant limitations and risks. The GPT-4 System Card includes insights from an in depth exploration of such risks conducted by OpenAI.

These are only just a few of the risks related to each models:

• Hallucination (the tendency to provide nonsensical or factually inaccurate content)
• Producing harmful content that violates OpenAI’s policies (e.g. hate speech, incitements to violence)
• Amplifying and perpetuating stereotypes of marginalized people
• Generating realistic disinformation intended to deceive

While ChatGPT and GPT-4 struggle with the identical limitations and risks, OpenAI has made special efforts, including extensive adversarial testing, to mitigate them for GPT-4. While that is encouraging, the GPT-4 System Card ultimately demonstrates how vulnerable ChatGPT was (and possibly still is). For a more detailed explanation of harmful unintended consequences, I like to recommend reading the GPT-4 System Card, which starts on page 38 of the GPT-4 Technical Report.

In this text, we review an important similarities and differences between ChatGPT and GPT-4, including their training methods, performance and capabilities, and limitations and risks.

While we all know much less concerning the model architecture and training methods behind GPT-4, it appears to be a refined version of ChatGPT that now accepts image and text inputs and claims to be safer, more accurate, and more creative. Unfortunately, we could have to take OpenAI’s word for it, as GPT-4 is just available as a part of the ChatGPT Plus subscription.

The table below illustrates an important similarities and differences between ChatGPT and GPT-4:

The race for creating essentially the most accurate and dynamic large language models has reached breakneck speed, with the discharge of ChatGPT and GPT-4 inside mere months of one another. Staying informed on the advancements, risks, and limitations of those models is crucial as we navigate this exciting but rapidly evolving landscape of huge language models.