For all their impressive capabilities, large language models (LLMs) often fall short when given difficult recent tasks that require complex reasoning skills.
While an accounting firm’s LLM might excel at summarizing financial reports, that very same model could fail unexpectedly if tasked with predicting market trends or identifying fraudulent transactions.
To make LLMs more adaptable, MIT researchers investigated how a certain training technique will be strategically deployed to spice up a model’s performance on unfamiliar, difficult problems.
They show that test-time training, a way that involves temporarily updating a few of a model’s inner workings during deployment, can result in a sixfold improvement in accuracy. The researchers developed a framework for implementing a test-time training strategy that uses examples of the brand new task to maximise these gains.
Their work could improve a model’s flexibility, enabling an off-the-shelf LLM to adapt to complex tasks that require planning or abstraction. This may lead to LLMs that will be more accurate in lots of applications that require logical deduction, from medical diagnostics to provide chain management.
“Real learning — what we did here with test-time training — is something these models can’t do on their very own after they’re shipped. They will’t gain recent skills or recuperate at a task. But now we have shown that when you push the model slightly bit to do actual learning, you see that vast improvements in performance can occur,” says Ekin Akyürek PhD ’25, lead writer of the study.
Akyürek is joined on the paper by graduate students Mehul Damani, Linlu Qiu, Han Guo, and Jyothish Pari; undergraduate Adam Zweiger; and senior authors Yoon Kim, an assistant professor of Electrical Engineering and Computer Science (EECS) and a member of the Computer Science and Artificial Intelligence Laboratory (CSAIL); and Jacob Andreas, an associate professor in EECS and a member of CSAIL. The research will likely be presented on the International Conference on Machine Learning.
Tackling hard domains
LLM users often try to enhance the performance of their model on a brand new task using a method called in-context learning. They feed the model a couple of examples of the brand new task as text prompts which guide the model’s outputs.
But in-context learning doesn’t all the time work for problems that require logic and reasoning.
The MIT researchers investigated how test-time training will be used at the side of in-context learning to spice up performance on these difficult tasks. Test-time training involves updating some model parameters — the inner variables it uses to make predictions — using a small amount of latest data specific to the duty at hand.
The researchers explored how test-time training interacts with in-context learning. They studied design selections that maximize the performance improvements one can coax out of a general-purpose LLM.
“We discover that test-time training is a much stronger type of learning. While simply providing examples can modestly boost accuracy, actually updating the model with those examples can result in significantly higher performance, particularly in difficult domains,” Damani says.
In-context learning requires a small set of task examples, including problems and their solutions. The researchers use these examples to create a task-specific dataset needed for test-time training.
To expand the scale of this dataset, they create recent inputs by barely changing the issues and solutions within the examples, reminiscent of by horizontally flipping some input data. They find that training the model on the outputs of this recent dataset results in the very best performance.
As well as, the researchers only update a small variety of model parameters using a method called low-rank adaption, which improves the efficiency of the test-time training process.
“This is very important because our method must be efficient if it’s going to be deployed in the true world. We discover you could get huge improvements in accuracy with a really small amount of parameter training,” Akyürek says.
Developing recent skills
Streamlining the method is essential, since test-time training is employed on a per-instance basis, meaning a user would wish to do that for every individual task. The updates to the model are only temporary, and the model reverts to its original form after making a prediction.
A model that typically takes lower than a minute to reply a question might take five or 10 minutes to offer a solution with test-time training, Akyürek adds.
“We wouldn’t wish to do that for all user queries, however it is helpful if you could have a really hard task that you need to the model to resolve well. There also may be tasks which can be too difficult for an LLM to resolve without this method,” he says.
The researchers tested their approach on two benchmark datasets of extremely complex problems, reminiscent of IQ puzzles. It boosted accuracy as much as sixfold over techniques that use only in-context learning.
Tasks that involved structured patterns or those which used completely unfamiliar forms of data showed the biggest performance improvements.
“For less complicated tasks, in-context learning may be OK. But updating the parameters themselves might develop a brand new skill within the model,” Damani says.
In the longer term, the researchers wish to use these insights toward the event of models that continually learn.
The long-term goal is an LLM that, given a question, can robotically determine if it needs to make use of test-time training to update parameters or if it could possibly solve the duty using in-context learning, after which implement the very best test-time training strategy without the necessity for human intervention.
This work is supported, partly, by the MIT-IBM Watson AI Lab and the National Science Foundation.
