Researchers at Alibaba’s Tongyi Lab have developed a brand new framework for self-evolving agents that create their very own training data by exploring their application environments. The framework, AgentEvolver, uses the knowledge and reasoning capabilities of huge language models for autonomous learning, addressing the high costs and manual effort typically required to assemble task-specific datasets.
Experiments show that in comparison with traditional reinforcement learning–based frameworks, AgentEvolver is more efficient at exploring its environment, makes higher use of information, and adapts faster to application environments. For the enterprise, this is important since it lowers the barrier to training agents for bespoke applications, making powerful, custom AI assistants more accessible to a wider range of organizations.
The high cost of coaching AI agents
Reinforcement learning has change into a serious paradigm for training LLMs to act as agents that may interact with digital environments and learn from feedback. Nevertheless, developing agents with RL faces fundamental challenges. First, gathering the crucial training datasets is usually prohibitively expensive, requiring significant manual labor to create examples of tasks, especially in novel or proprietary software environments where there aren’t any available off-the-shelf datasets.
Second, the RL techniques commonly used for LLMs require the model to run through a large variety of trial-and-error attempts to learn effectively. This process is computationally costly and inefficient. In consequence, training capable LLM agents through RL stays laborious and expensive, limiting their deployment in custom enterprise settings.
How AgentEvolver works
The predominant idea behind AgentEvolver is to offer models greater autonomy in their very own learning process. The researchers describe it as a “self-evolving agent system” designed to “achieve autonomous and efficient capability evolution through environmental interaction.” It uses the reasoning power of an LLM to create a self-training loop, allowing the agent to constantly improve by directly interacting with its goal environment with no need predefined tasks or reward functions.
“We envision an agent system where the LLM actively guides exploration, task generation, and performance refinement,” the researchers wrote in their paper.
The self-evolution process is driven by three core mechanisms that work together.
The primary is self-questioning, where the agent explores its environment to find the boundaries of its functions and discover useful states. It’s like a brand new user clicking around an application to see what’s possible. Based on this exploration, the agent generates its own diverse set of tasks that align with a user’s general preferences. This reduces the necessity for handcrafted datasets and allows the agent and its tasks to co-evolve, progressively enabling it to handle more complex challenges.
In response to Yunpeng Zhai, researcher at Alibaba and co-author of the paper, who spoke to VentureBeat, the self-questioning mechanism effectively turns the model from a “data consumer into an information producer,” dramatically reducing the time and price required to deploy an agent in a proprietary environment.
The second mechanism is self-navigating, which improves exploration efficiency by reusing and generalizing from past experiences. AgentEvolver extracts insights from each successful and unsuccessful attempts and uses them to guide future actions. For instance, if an agent tries to make use of an API function that doesn't exist in an application, it registers this as an experience and learns to confirm the existence of functions before attempting to make use of them in the long run.
The third mechanism, self-attributing, enhances learning efficiency by providing more detailed feedback. As an alternative of only a final success or failure signal (a standard practice in RL that may end up in sparse rewards), this mechanism uses an LLM to evaluate the contribution of every individual motion in a multi-step task. It retrospectively determines whether each step contributed positively or negatively to the ultimate final result, giving the agent fine-grained feedback that accelerates learning.
That is crucial for regulated industries where how an agent solves an issue is as essential because the result. “As an alternative of rewarding a student just for the ultimate answer, we also evaluate the clarity and correctness of every step of their reasoning,” Zhai explained. This improves transparency and encourages the agent to adopt more robust and auditable problem-solving patterns.
“By shifting the training initiative from human-engineered pipelines to LLM-guided self-improvement, AgentEvolver establishes a brand new paradigm that paves the way in which toward scalable, cost-effective, and continually improving intelligent systems,” the researchers state.
The team has also developed a practical, end-to-end training framework that integrates these three mechanisms. A key a part of this foundation is the Context Manager, a component that controls the agent's memory and interaction history. While today's benchmarks test a limited variety of tools, real enterprise environments can involve 1000’s of APIs.
Zhai acknowledges it is a core challenge for the sector, but notes that AgentEvolver was designed to be prolonged. “Retrieval over extremely large motion spaces will all the time introduce computational challenges, but AgentEvolver’s architecture provides a transparent path toward scalable tool reasoning in enterprise settings,” he said.
A more efficient path to agent training
To measure the effectiveness of their framework, the researchers tested it on AppWorld and BFCL v3, two benchmarks that require agents to perform long, multi-step tasks using external tools. They used models from Alibaba’s Qwen2.5 family (7B and 14B parameters) and compared their performance against a baseline model trained with GRPO, a well-liked RL technique used to develop reasoning models like DeepSeek-R1.
The outcomes showed that integrating all three mechanisms in AgentEvolver led to substantial performance gains. For the 7B model, the common rating improved by 29.4%, and for the 14B model, it increased by 27.8% over the baseline. The framework consistently enhanced the models' reasoning and task-execution capabilities across each benchmarks. Essentially the most significant improvement got here from the self-questioning module, which autonomously generates diverse training tasks and directly addresses the info scarcity problem.
The experiments also demonstrated that AgentEvolver can efficiently synthesize a big volume of high-quality training data. The tasks generated by the self-questioning module proved diverse enough to attain good training efficiency even with a small amount of information.
For enterprises, this provides a path to creating agents for bespoke applications and internal workflows while minimizing the necessity for manual data annotation. By providing high-level goals and letting the agent generate its own training experiences, organizations can develop custom AI assistants more simply and cost-effectively.
“This mix of algorithmic design and engineering pragmatics positions AgentEvolver as each a research vehicle and a reusable foundation for constructing adaptive, tool-augmented agents,” the researchers conclude.
Looking ahead, the final word goal is far larger. “A very ‘singular model’ that may drop into any software environment and master it overnight is definitely the holy grail of agentic AI,” Zhai said. “We see AgentEvolver as a crucial step in that direction.” While that future still requires breakthroughs in model reasoning and infrastructure, self-evolving approaches are paving the way in which.
