It’s necessary to notice here that prompt injection has not yet caused any catastrophes, or a minimum of none which were publicly reported. But now that there are likely a whole lot of hundreds of OpenClaw agents buzzing across the web, prompt injection might begin to seem like a far more appealing strategy for cybercriminals. “Tools like this are incentivizing malicious actors to attack a wider population,” Papernot says.
Constructing guardrails
The term “prompt injection” was coined by the favored LLM blogger Simon Willison in 2022, a few months before ChatGPT was released. Even back then, it was possible to discern that LLMs would introduce a very latest kind of security vulnerability once they got here into widespread use. LLMs can’t tell apart the instructions that they receive from users and the information that they use to perform those instructions, equivalent to emails and web search results—to an LLM, they’re all just text. So if an attacker embeds just a few sentences in an email and the LLM mistakes them for an instruction from its user, the attacker can get the LLM to do anything it wants.
Prompt injection is a troublesome problem, and it doesn’t appear to be going away anytime soon. “We don’t really have a silver-bullet defense at once,” says Dawn Song, a professor of computer science at UC Berkeley. But there’s a sturdy academic community working on the issue, and so they’ve give you strategies that would eventually make AI personal assistants secure.
Technically speaking, it is feasible to make use of OpenClaw today without risking prompt injection: Just don’t connect it to the web. But restricting OpenClaw from reading your emails, managing your calendar, and doing online research defeats much of the aim of using an AI assistant. The trick of protecting against prompt injection is to stop the LLM from responding to hijacking attempts while still giving it room to do its job.
One strategy is to coach the LLM to disregard prompt injections. A serious a part of the LLM development process, called post-training, involves taking a model that knows find out how to produce realistic text and turning it right into a useful assistant by “rewarding” it for answering questions appropriately and “punishing” it when it fails to accomplish that. These rewards and punishments are metaphorical, however the LLM learns from them as an animal would. Using this process, it’s possible to coach an LLM not to answer specific examples of prompt injection.
But there’s a balance: Train an LLM to reject injected commands too enthusiastically, and it may also begin to reject legitimate requests from the user. And since there’s a fundamental element of randomness in LLM behavior, even an LLM that has been very effectively trained to withstand prompt injection will likely still slip up every every now and then.
One other approach involves halting the prompt injection attack before it ever reaches the LLM. Typically, this involves using a specialized detector LLM to find out whether or not the information being sent to the unique LLM comprises any prompt injections. In a recent study, nonetheless, even the best-performing detector completely failed to select up on certain categories of prompt injection attack.
The third strategy is more complicated. Quite than controlling the inputs to an LLM by detecting whether or not they contain a prompt injection, the goal is to formulate a policy that guides the LLM’s outputs—i.e., its behaviors—and prevents it from doing anything harmful. Some defenses on this vein are quite easy: If an LLM is allowed to email only just a few pre-approved addresses, for instance, then it definitely won’t send its user’s bank card information to an attacker. But such a policy would prevent the LLM from completing many useful tasks, equivalent to researching and reaching out to potential skilled contacts on behalf of its user.
