We’ve trained language models which can be a lot better at following user intentions than GPT-3 while also making them more truthful and fewer toxic, using techniques developed through our alignment research. These InstructGPT models, that are trained with humans within the loop, at the moment are deployed because the default language models on our API.