The flash crash might be probably the most well-known example of the hazards raised by agents—automated systems which have the ability to take actions in the actual world, without human oversight. That power is the source of their value; the agents that supercharged the flash crash, for instance, could trade far faster than any human. Nevertheless it’s also why they may cause a lot mischief. “The nice paradox of agents is that the very thing that makes them useful—that they’re in a position to accomplish a variety of tasks—involves freely giving control,” says Iason Gabriel, a senior staff research scientist at Google DeepMind who focuses on AI ethics.
“If we proceed on the present path … we’re mainly playing Russian roulette with humanity.”
Yoshua Bengio, professor of computer science, University of Montreal
Agents are already all over the place—and have been for a lot of many years. Your thermostat is an agent: It robotically turns the heater on or off to maintain your own home at a particular temperature. So are antivirus software and Roombas. Like high-frequency traders, that are programmed to purchase or sell in response to market conditions, these agents are all built to perform specific tasks by following prescribed rules. Even agents which are more sophisticated, comparable to Siri and self-driving cars, follow prewritten rules when performing a lot of their actions.
But in recent months, a brand new class of agents has arrived on the scene: ones built using large language models. Operator, an agent from OpenAI, can autonomously navigate a browser to order groceries or make dinner reservations. Systems like Claude Code and Cursor’s Chat feature can modify entire code bases with a single command. Manus, a viral agent from the Chinese startup Butterfly Effect, can construct and deploy web sites with little human supervision. Any motion that could be captured by text—from playing a video game using written commands to running a social media account—is potentially throughout the purview of the sort of system.
LLM agents don’t have much of a track record yet, but to listen to CEOs tell it, they’ll transform the economy—and shortly. OpenAI CEO Sam Altman says agents might “join the workforce” this 12 months, and Salesforce CEO Marc Benioff is aggressively promoting Agentforce, a platform that permits businesses to tailor agents to their very own purposes. The US Department of Defense recently signed a contract with Scale AI to design and test agents for military use.
Scholars, too, are taking agents seriously. “Agents are the following frontier,” says Dawn Song, a professor of electrical engineering and computer science on the University of California, Berkeley. But, she says, “to ensure that us to actually profit from AI, to truly [use it to] solve complex problems, we’d like to work out the right way to make them work safely and securely.”
PATRICK LEGER
That’s a tall order. Like chatbot LLMs, agents could be chaotic and unpredictable. Within the near future, an agent with access to your checking account could assist you to manage your budget, however it may additionally spend all of your savings or leak your information to a hacker. An agent that manages your social media accounts could alleviate among the drudgery of maintaining a web-based presence, however it may additionally disseminate falsehoods or spout abuse at other users.
Yoshua Bengio, a professor of computer science on the University of Montreal and considered one of the so-called “godfathers of AI,” is amongst those concerned about such risks. What worries him most of all, though, is the likelihood that LLMs could develop their very own priorities and intentions—after which act on them, using their real-world abilities. An LLM trapped in a chat window can’t do much without human assistance. But a robust AI agent could potentially duplicate itself, override safeguards, or prevent itself from being shut down. From there, it’d do whatever it wanted.
As of now, there’s no foolproof approach to guarantee that agents will act as their developers intend or to forestall malicious actors from misusing them. And though researchers like Bengio are working hard to develop latest safety mechanisms, they could not have the ability to maintain up with the rapid expansion of agents’ powers. “If we proceed on the present path of constructing agentic systems,” Bengio says, “we’re mainly playing Russian roulette with humanity.”
