3. Permissions by design: Bind tools to tasks, to not models
A standard anti-pattern is to offer the model a long-lived credential and hope prompts keep it polite. SAIF and NIST argue the alternative: credentials and scopes needs to be certain to tools and tasks, rotated frequently, and auditable. Agents then request narrowly scoped capabilities through those tools.
In practice, that appears like: “finance-ops-agent may read, but not write, certain ledgers without CFO approval.”
Control data and behavior
These steps gate inputs, outputs, and constrain behavior.
4. Inputs, memory, and RAG: Treat external content as hostile until proven otherwise
Most agent incidents start with sneaky data: a poisoned web page, PDF, email, or repository that smuggles adversarial instructions into the system. OWASP’s prompt-injection cheat sheet and OpenAI’s own guidance each insist on strict separation of system instructions from user content and on treating unvetted retrieval sources as untrusted.
Operationally, gate before anything enters retrieval or long-term memory: latest sources are reviewed, tagged, and onboarded; persistent memory is disabled when untrusted context is present; provenance is attached to every chunk.
5. Output handling and rendering: Nothing executes “simply because the model said so”
Within the Anthropic case, AI-generated exploit code and credential dumps flowed straight into motion. Any output that could cause a side effect needs a validator between the agent and the actual world. OWASP’s insecure output handling category is explicit on this point, as are browser security best practices around origin boundaries.
