Step-by-step guide for GPT-3 fine-tuning

For the reason that end of 2022, the launch of ChatGPT by OpenAI has been considered by lots of us to be the iPhone moment of AI. Nonetheless, OpenAI’s chatbot isn’t the primary generative AI text machine learning model and follows the launch of GPT-3, which was launched two years earlier.

OpenAI provides us with a ready-to-use GPT-3 trained model. Moreover, a selected task might be fine-tuned on a smaller dataset. For instance, suppose you desire to create an email response generator specific to your organization. First, you could collect a considerable amount of data about your particular business domain, comparable to customer email inquiries and responses. You possibly can then use this data to fine-tune GPT-3 to learn your organization’s specific language patterns and phrases. By fine-tuning GPT-3, making a highly customized and specialized email response generator is feasible, specifically tailored to the language patterns and words utilized in a specific business domain.

On this blog post, I’ll show you find out how to fine-tune GPT-3. We’ll do that with python code and without assuming prior knowledge about GPT-3.

Unlike the GPT-2 model available as an example on Hugging Face (when this blog is written), we don’t have direct access to the GPT-3 model. Due to this fact, you first must get an API key from OpenAI and install the Python package openai, which might be quickly done via pip.

For the API key from OpenAI:

The secret’s an extended string of characters starting with ‘sk-’. Be sure you retain it secret! Once you’ve your key, a straightforward solution to get access to your secret’s to do the next in your terminal: (personally, for simplicity, I put it in my .bachrc):

export OPENAI_API_KEY=sk-t59pgejhtrff5(...)

Using GPT-3 models has a value. We want credits. On the time of this writing, while you create a recent account, you get free credits to check out the tool. I don’t know if this may proceed…

Now that we have now our key and Python package, it’s time to think concerning the data we’d like to fine-tune. First, we’d like a file of examples for fine-tuning, where each example is a prompt followed by the suitable completion.

We’ll construct a tool for this demo to create descriptions of imaginary superheroes. Ultimately, the tool will receive the age, gender, and power of the superhero, and it can robotically produce an outline of our superhero.

In the next example, after fine-tuning the model, all we have now to say is ’40, woman, Healing ->’, and we are going to robotically get an outline from the model.

That is what it’s all about!! 😃

In some situations, you might have a knowledge set you desire to use for fine-tuning. But since I don’t have one, let’s see find out how to create an artificial data set with the outline of the superheroes directly from GPT-3. The next code will give me a CSV file with examples of prompts and the corresponding completions.

import os
import openai
import pandas as pd
openai.api_key = os.getenv("OPENAI_API_KEY")
l_age = ['18', '20', '30', '40', '50', '60', '90']
l_gender = ['man', 'woman']
l_power = ['invisibility', 'read in the thoughts', 'turning lead into gold', 'immortality', 'telepathy', 'teleport', 'flight'] 
f_prompt = "Imagine an entire and detailed description of a {age}-year-old {gender} fictional character who has the superpower of {power}. Write out the complete description in a maximum of 100 words in great detail:"
f_sub_prompt = "{age}, {gender}, {power}"
df = pd.DataFrame()
for age in l_age:
for gender in l_gender:
for power in l_power:
for i in range(3): ## 3 times each
prompt = f_prompt.format(age=age, gender=gender, power=power)
sub_prompt = f_sub_prompt.format(age=age, gender=gender, power=power)
print(sub_prompt)
response = openai.Completion.create(
model="text-davinci-003",
prompt=prompt,
temperature=1,
max_tokens=500,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)
finish_reason = response['choices'][0]['finish_reason']
response_txt = response['choices'][0]['text']
new_row = {
'age':age, 
'gender':gender, 
'power':power, 
'prompt':prompt, 
'sub_prompt':sub_prompt, 
'response_txt':response_txt, 
'finish_reason':finish_reason}
new_row = pd.DataFrame([new_row])
df = pd.concat([df, new_row], axis=0, ignore_index=True)
df.to_csv("out_openai_completion.csv")

Let’s have a look at how this code works 🧐.

The variable f_prompt incorporates the next sentence where {age}, {gender}, and {power} are missing.

Imagine an entire and detailed description of a {age}-year-old {gender} fictional character who has the superpower of {power}. Write out the complete description in a maximum of 100 words in great detail:

In the primary three for loops of the code, we iterate over different values of {age}, {gender}, and {power}. At each step of the loop, we replace the three missing variables with different values. Then we use the openai.Completion.create function to ask GPT to generate a response to our prompt.

An important parameters of this function are

• model: The model used to generate the response. OpenAI offers 4 standard GPT-3 models (ada, babbage, curie, or davinci) that fluctuate in size … and price of use. Here it’s davinci — the largest model.
• prompt: The prompt that we wish to satisfy with GPT-3.
• temperature: The temperature is a number between 0 and 1 and controls how much randomness is within the output. We set the temperature to the utmost to permit the model to be as creative as possible in creating the response.
• max_tokens: Defines the utmost length of the response.

At the top of this script, we have now a Pandas table stored within the file out_openai_completion.csv. The 2 primary columns on this table that interest us are sub_prompt and response_txt.

• The sub_prompt might be for instance ‘18, man, invisibility’. It incorporates the three values that were replaced, separated by commas.
• The response_txt incorporates the output of the GPT model.

The next code retrieves the previously created file out_openai_completion.csv and uses openai to fine-tune a GPT-3 model.

import pandas as pd
import openai
import subprocess
df = pd.read_csv("out_openai_completion.csv")
prepared_data = df.loc[:,['sub_prompt','response_txt']]
prepared_data.rename(columns={'sub_prompt':'prompt', 'response_txt':'completion'}, inplace=True)
prepared_data.to_csv('prepared_data.csv',index=False)
## prepared_data.csv --> prepared_data_prepared.json
subprocess.run('openai tools fine_tunes.prepare_data --file prepared_data.csv --quiet'.split())
## Start fine-tuning
subprocess.run('openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model davinci --suffix "SuperHero"'.split())

Let’s take the time to know this code as well 🤓!

First, the content of the file out_openai_completion.csv is loaded into the info frame df. As a reminder, in our task, if the user enters ’40, female, healing’, we wish to have an outline of a 40-year-old female character with the facility of healing. To perform fine-tuning, it’s essential to supply GPT with examples of what the user might type and the corresponding desired response. In the info frame df, the columns sub_prompt and response_txt contain examples of input with the corresponding desired response. Within the code above, we first extract these two colons after which rename them to prompt and completion, respectively. The resulting data frame is stored in a recent file prepared_data.csv containing only these two columns.

This file prepared_data.csv looks like this:

prompt,completion
"18, man, invisibility","He looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His construct is athletic and powerful, but not muscular. He often wears clothing that makes him mix in together with his surroundings, with a purpose to stay as inconspicuous as possible.He has the power of invisibility, which he can use to turn out to be an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, capable of even make sounds vanish, allowing him to go unnoticed. His invisibility power is his biggest asset, and he uses it to guard those he cares about."
"18, man, invisibility","18 year-old Jackson is a strong young man with the incredible power of invisibility. He stands at 6'2 and has a powerful athletic frame with wavy brown hair, startling blue eyes and a warm and mild smile. His strength and agility make him especially good at using his ability. He can turn out to be invisible straight away, moving through partitions and other obstacles without being detected. What's more, he can move objects together with his mind and use his power to guard those he loves. His power is a blessing and a curse, as it could actually be abused to do harm. Jackson is a brave and noble one that uses his power to do good and make the world a greater place."
"18, man, invisibility","Brandon is an 18-year-old of average construct, standing roughly 5'10 with an inquisitive look. He has naturally wavy chestnut brown hair and vibrant blue eyes. His demeanor is normally quite serious, but he also has a straightforward and mild smile. He has a natural gift of invisibility, which he uses to guard himself and his family from harm. He's an inquisitive and artistic young man who uses his gift to explore the world, observe people, and uncover the reality. His consistent practice of mindfulness helps him remain unseen, even when his emotions are visible. His intelligence and chronic passion for truth drives him to assist those in need."
"18, man, read within the thoughts","This 18-year-old man has a muscular stature and long, ash blonde hair. His vibrant blue eyes are full of information and power, hinting on the strange ability he has - he can read minds. From time to time, strange sparks of electricity streak from his eyes when he concentrates hard enough. He is commonly calm and picked up, but when provoked has the power to mix his learning of your thoughts together with his immense strength. His wit, intelligence and empathy are weapons which might be often underestimated - yet, very lethal."
"18, man, read within the thoughts","He's an 18-year-old man with short, cropped dark hair and vibrant green eyes. His usual outfit consists of an olive green hoodie, blue jeans and purple trainers. He's relaxed, confident and all the time reads the atmosphere well. Certainly one of his great skills is read within the thoughts of others. By doing so, he's capable of understand people more easily and relate to their concerns out of the blue. He all the time uses his superpower with the utmost dignity and respect, never using it to control. He uses it to hunt the reality, discover people's emotions and gain insight into their perspectives. He's the type of individual that could make anyone feel comfortable and is rarely afraid of a challenge. His superpower is merely an extension of his already charismatic disposition."
(...)

The Python subprocess.run() function runs a command as a subprocess. It is commonly used to execute external programs as in the event that they were run in a terminal.

We use subprocess.run() to execute ‘openai tools fine_tunes.prepare_data’. This function takes the input file prepared_data.csv, checks that the info are correct, and produces a JSONL file called prepared_data_prepared.jsonl. A JSONL file is a format that stores each JSON object on a separate line. JSONL files contain a sequence of JSON objects, each separated by a newline character.

Note that we have now added the choice “- – quiet” to robotically accept all recommendations made by ‘openai tools fine_tunes.prepare_data’. For example, it suggests adding a ‘–>’ to the top of all prompts and adding a token END to the top of every response.

The primary lines of this JSONL file seem like this:

{"prompt":"18, man, invisibility ->","completion":" nnHe looks to be in his late teens, with dark eyes and unruly black hair which reach his shoulders. His construct is athletic and powerful, but not muscular. He often wears clothing that makes him mix in together with his surroundings, with a purpose to stay as inconspicuous as possible.nnHe has the power of invisibility, which he can use to turn out to be an invisible observer or to make himself or objects around him disappear from view. He has honed his skills to make himself undetectable, capable of even make sounds vanish, allowing him to go unnoticed. His invisibility power is his biggest asset, and he uses it to guard those he cares about. END"}
{"prompt":"18, man, invisibility ->","completion":" nn18 year-old Jackson is a strong young man with the incredible power of invisibility. He stands at 6'2 and has a powerful athletic frame with wavy brown hair, startling blue eyes and a warm and mild smile. His strength and agility make him especially good at using his ability. He can turn out to be invisible straight away, moving through partitions and other obstacles without being detected. What's more, he can move objects together with his mind and use his power to guard those he loves. His power is a blessing and a curse, as it could actually be abused to do harm. Jackson is a brave and noble one that uses his power to do good and make the world a greater place. END"}
(...)

The fine-tuning of the GPT-3 model is absolutely achieved within the second subprocess.run(), where openai api fine_tunes.create is executed. On this function, we start by giving the name of the JSONL file created just before. You’ll then need to pick the model you want to fine-tune. OpenAI offers 4 principal models with different performance levels suitable for various tasks. Davinci is probably the most powerful model, and Ada is the fastest. Davinci can be the most costly model 😨.

For the reason that purpose of my model is to create descriptions of superheroes, we give my recent model the suffix “Superhero”.

And that’s it 😉 After a number of minutes, you should have a fine-tuned model able to use 🌟.

There are alternative ways to make use of a model for completion. Mainly via the Playground provided by OpenAI or via programming languages like Python.

The easiest method might be to make use of the playground.

It’s now time to ask our model for a recent prediction. We’ll ask to explain an 18-year-old male character who really has an unnecessary power 😉 We’ll ask to explain a personality who has the facility… ‘can eat quite a bit’… and see what happens… 😆

Not so bad 😅

Do you desire to do it in Python? Easy! Click on ‘View code’ at the highest right of the screen.

In our case, in ‘View code’ we have now this:

import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")
response = openai.Completion.create(
model="davinci:ft-klhm:superhero-2023-02-01-14-56-48",
prompt="18, Man, can eat quite a bit ->n",
temperature=0.7,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0,
stop=["END"]
)

You simply should copy and paste it 👍.

On this blog, we have now seen find out how to generate synthetic data to refine our model and find out how to do this fine-tuning. We now have used a use case of making a superhero, but the identical method might be used for any use case you might have. An important thing is to have enough quality examples with prompts and desired responses.

Please consider following me in the event you want to not sleep up to now with my latest publications and increase the visibility of this blog.