Whether you’re a scientist brainstorming research ideas or a CEO hoping to automate a task in human resources or finance, you’ll find that artificial intelligence tools have gotten the assistants you didn’t know you needed. Particularly, many professionals are tapping into the skills of semi-autonomous software systems called AI agents, which may call on AI at specific points to unravel problems and complete tasks.
AI agents are particularly effective after they use large language models (LLMs) because those systems are powerful, efficient, and adaptable. One solution to program such technology is by describing in code what you wish your system to do (the “workflow”), including when it should use an LLM. In case you were a software company attempting to revamp your old codebase to make use of a more modern programming language for higher optimizations and safety, you may construct a system that uses an LLM to translate the codebase one file at a time, testing each file as you go.
But what happens when LLMs make mistakes? You’ll want the agent to backtrack to make one other attempt, incorporating lessons it learned from previous mistakes. Coding this up can take as much effort as implementing the unique agent; in case your system for translating a codebase contained hundreds of lines of code, you then’d be making hundreds of lines of code changes or additions to support the logic for backtracking when LLMs make mistakes.
To save lots of programmers effort and time, researchers with MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and Asari AI have developed a framework called “EnCompass.”
With EnCompass, you not should make these changes yourself. As a substitute, when EnCompass runs your program, it mechanically backtracks if LLMs make mistakes. EnCompass may also make clones of this system runtime to make multiple attempts in parallel in the hunt for the very best solution. In full generality, EnCompass searches over the various possible paths your agent could take because of this of the various possible outputs of all of the LLM calls, searching for the trail where the LLM finds the very best solution.
Then, all you have got to do is to annotate the locations where it’s possible you’ll wish to backtrack or clone this system runtime, in addition to record any information which may be useful to the strategy used to go looking over the various possible execution paths of your agent (the search strategy). You possibly can then individually specify the search strategy — you would either use one which EnCompass provides out of the box or, if desired, implement your individual custom search strategy.
“With EnCompass, we’ve separated the search strategy from the underlying workflow of an AI agent,” says lead creator Zhening Li ’25, MEng ’25, who’s an MIT electrical engineering and computer science (EECS) PhD student, CSAIL researcher, and research consultant at Asari AI. “Our framework lets programmers easily experiment with different search strategies to seek out the one which makes the AI agent perform the very best.”
EnCompass was used for agents implemented as Python programs that decision LLMs, where it demonstrated noticeable code savings. EnCompass reduced coding effort for implementing search by as much as 80 percent across agents, equivalent to an agent for translating code repositories and for locating transformation rules of digital grids. In the longer term, EnCompass could enable agents to tackle large-scale tasks, including managing massive code libraries, designing and carrying out science experiments, and creating blueprints for rockets and other hardware.
Branching out
When programming your agent, you mark particular operations — equivalent to calls to an LLM — where results may vary. These annotations are called “branchpoints.” In case you imagine your agent program as generating a single plot line of a story, then adding branchpoints turns the story right into a choose-your-own-adventure story game, where branchpoints are locations where the plot branches into multiple future plot lines.
You possibly can then specify the strategy that EnCompass uses to navigate that story game, in the hunt for the very best possible ending to the story. This may include launching parallel threads of execution or backtracking to a previous branchpoint while you get stuck in a dead end.
Users may also plug-and-play a number of common search strategies provided by EnCompass out of the box, or define their very own custom strategy. For instance, you would go for Monte Carlo tree search, which builds a search tree by balancing exploration and exploitation, or beam search, which keeps the very best few outputs from every step. EnCompass makes it easy to experiment with different approaches to seek out the very best technique to maximize the likelihood of successfully completing your task.
The coding efficiency of EnCompass
So just how code-efficient is EnCompass for adding search to agent programs? Based on researchers’ findings, the framework drastically cut down how much programmers needed so as to add to their agent programs so as to add search, helping them experiment with different strategies to seek out the one which performs the very best.
For instance, the researchers applied EnCompass to an agent that translates a repository of code from the Java programming language, which is usually used to program apps and enterprise software, to Python. They found that implementing search with EnCompass — mainly involving adding branchpoint annotations and annotations that record how well each step did — required 348 fewer lines of code (about 82 percent) than implementing it by hand. In addition they demonstrated how EnCompass enabled them to simply check out different search strategies, identifying the very best technique to be a two-level beam search algorithm, achieving an accuracy boost of 15 to 40 percent across five different repositories at a search budget of 16 times the LLM calls made by the agent without search.
“As LLMs turn out to be a more integral a part of on a regular basis software, it becomes more necessary to grasp tips on how to efficiently construct software that leverages their strengths and works around their limitations,” says co-author Armando Solar-Lezama, who’s an MIT professor of EECS and CSAIL principal investigator. “EnCompass is a crucial step in that direction.”
The researchers add that EnCompass targets agents where a program specifies the steps of the high-level workflow; the present iteration of their framework is less applicable to agents which might be entirely controlled by an LLM. “In those agents, as a substitute of getting a program that specifies the steps after which using an LLM to perform those steps, the LLM itself decides every part,” says Li. “There is no such thing as a underlying programmatic workflow, so you’ll be able to execute inference-time search on regardless of the LLM invents on the fly. On this case, there’s less need for a tool like EnCompass that modifies how a program executes with search and backtracking.”
Li and his colleagues plan to increase EnCompass to more general search frameworks for AI agents. In addition they plan to check their system on more complex tasks to refine it for real-world uses, including at firms. What’s more, they’re evaluating how well EnCompass helps agents work with humans on tasks like brainstorming hardware designs or translating much larger code libraries. For now, EnCompass is a robust constructing block that allows humans to tinker with AI agents more easily, improving their performance.
“EnCompass arrives at a timely moment, as AI-driven agents and search-based techniques are starting to reshape workflows in software engineering,” says Carnegie Mellon University Professor Yiming Yang, who wasn’t involved within the research. “By cleanly separating an agent’s programming logic from its inference-time search strategy, the framework offers a principled solution to explore how structured search can enhance code generation, translation, and evaluation. This abstraction provides a solid foundation for more systematic and reliable search-driven approaches to software development.”
Li and Solar-Lezama wrote the paper with two Asari AI researchers: Caltech Professor Yisong Yue, an advisor at the corporate; and senior creator Stephan Zheng, who’s the founder and CEO. Their work was supported by Asari AI.
The team’s work was presented on the Conference on Neural Information Processing Systems (NeurIPS) in December.
