The model mastered using tools reminiscent of calculators, calendars, or Wikipedia search queries across many downstream tasks.

I recently began an AI-focused educational newsletter, that already has over 150,000 subscribers. TheSequence is a no-BS (meaning no hype, no news etc) ML-oriented newsletter that takes 5 minutes to read. The goal is to maintain you up up to now with machine learning projects, research papers and ideas. Please give it a try by subscribing below:

Today’s large language models have made remarkable strides in performing a spread of natural language processing tasks, displaying a spread of emergent capabilities. Nonetheless, these models have certain inherent limitations that may only be partially mitigated by increasing their size. These limitations include an inability to access recent events, a bent to fabricate information, difficulties in processing low-resource languages, a scarcity of mathematical proficiency, and an ignorance of the passage of time. One promising approach to beat these limitations is to equip language models with the flexibility to make use of external tools reminiscent of search engines like google, calculators, or calendars. Nonetheless, current solutions either require extensive human annotations or are restricted to specific tasks, hindering wider adoption. A number of days ago, Meta AI published a research paper detailing Toolformer, a novel model that learns to make use of tools in a self-supervised manner without the necessity for human annotations.

Meta AI’s approach with Toolformer relies on the concept of in-context learning and the generation of datasets from scratch. Given just just a few examples of how an API could be used, Toolformer annotates a big language modeling dataset with potential API calls. Through a self-supervised loss, the model determines which API calls are useful in predicting future tokens and fine-tunes itself accordingly. With this approach, language models can learn to regulate a wide range of tools and to make informed decisions on when and the right way to use them. Toolformer allows the model to retain its generality and to independently resolve when and the right way to use various tools, enabling a more comprehensive utilization of tools that just isn’t tied to specific tasks.

The core idea behind Toolformer is to reinforce a language model (M) with the flexibility to make use of different tools via API calls. The inputs and outputs for every API are represented as text sequences, which enables the mixing of API calls into any text using special tokens.

For the training, Meta AI used a dataset of API calls represented as a tuple (ac, ic), where ac is the name of the API, and it’s the input. Given an API call (ac, ic) with a corresponding result (r), the linearized sequences of the API call without and with the result are denoted as e(ac, ic) and e(ac, ic, r), respectively. The dataset is step one to convert the dataset of plain texts into an augmented dataset by inserting API calls. This is finished in three steps: sampling potential API calls, executing the API calls and filtering the API calls based on their usefulness in predicting future tokens.

After filtering the API calls, they’re merged and interleaved with the unique inputs to form the augmented dataset. The language model is then finetuned on this augmented dataset, allowing it to make its own decisions on when and the right way to use each tool based by itself feedback.

Within the inference stage, the model generates text as usual until it encounters the “!” token, indicating the necessity for an API response. The suitable API is then called to acquire the response, and the decoding process continues after inserting the response and the token.

The researchers are investigating various tools to handle the constraints of standard language models (LMs). The one requirements for these tools are that their inputs and outputs could be represented as text sequences and that the researchers can obtain just a few examples of the right way to use them. The five tools being explored are a question-answering system, a Wikipedia search engine, a calculator, a calendar, and a machine translation system.

1. The question-answering system relies on one other LM that may answer easy factual questions.

2. The calculator can perform basic arithmetic operations and returns results rounded to 2 decimal places.

3. The Wikipedia search engine returns short text snippets from Wikipedia based on a search term.

4. the machine translation system can translate phrases from any language into English.

5. The calendar returns the present date without taking any input, providing a temporal context for predictions that require an awareness of time.

The Toolformer implementation relies on a finetuned version of GPT-J, which only uses 6.7 billion parameters. The model was in a position to outperform GPT-3 and GPT-J across several benchmarks.

The ideas behind Toolformer represent a latest frontier for LLMs by which they usually are not only in a position to perform sophisticated language tasks but complement them with access to tools and APIs. Can’t wait to see Meta AI expand on these ideas.