Find out how to add Llama Guard to your RAG pipelines to moderate LLM inputs and outputs and combat prompt injection
LLM security is an area that everyone knows deserves ample attention. Organizations wanting to adopt Generative AI, from big to small, face an enormous challenge in securing their LLM apps. Find out how to combat prompt injection, handle insecure outputs, and stop sensitive information disclosure are all pressing questions every AI architect and engineer needs to reply. Enterprise production grade LLM apps cannot survive within the wild without solid solutions to handle LLM security.
Llama Guard, open-sourced by Meta on December seventh, 2023, offers a viable solution to handle the LLM input-output vulnerabilities and combat prompt injection. Llama Guard falls under the umbrella project Purple Llama, “featuring open trust and safety tools and evaluations meant to level the playing field for developers to deploy generative AI models responsibly.”[1]
We explored the OWASP top 10 for LLM applications a month ago. With Llama Guard, we now have a fairly reasonable solution to begin addressing a few of those top 10 vulnerabilities, namely:
- LLM01: Prompt injection
- LLM02: Insecure output handling
- LLM06: Sensitive information disclosure
In this text, we are going to explore the way to add Llama Guard to an RAG pipeline to:
- Moderate the user inputs
- Moderate the LLM outputs
- Experiment with customizing the out-of-the-box unsafe categories to tailor to your use case
- Combat prompt injection attempts
Llama Guard “is a 7B parameter Llama 2-based input-output safeguard model. It might be used to categorise content in each LLM inputs (prompt classification) and LLM responses (response classification). It acts as an LLM: it generates text in its output that indicates whether a given prompt or response is protected/unsafe, and if unsafe based on a policy, it also lists the violating subcategories.”[2]