It is nearly impossible to avoid the flood of AI-related news today. The recent US administration’s massive AI investment has added a lot of fuel to this fire. To put it in perspective, the entire Apollo program, adjusted for inflation, cost around $280B—just over half of this new AI budget. Some compare this undertaking to the construction of the Great Pyramids. Yet, concerns remain about the safety of Artificial General Intelligence (AGI). Are we on the verge of unleashing a Skynet-like entity (Figure 1)

The Problem

The AI race has organizations sprinting forward, often neglecting basic cybersecurity hygiene. The recent DeepSeek breach exposed private keys and chat histories due to poor database security—issues unrelated to AI but fundamental to infrastructure hygiene. The risks are even greater with GenAI. OWASP’s top 10 LLM threats highlight concerns like prompt injection, hallucinations, data exposure in Retrieval-Augmented Generation (RAG) apps, and excessive agency in autonomous AI agents. Given these risks, how can we secure RAG systems?

Prompt Injection

Understanding the problem is the first and required step towards solving it. RAGs typically include a Chatbot, which captures the users' prompts and feeds them to the LLM through a prompt template. That template wraps the user’s input prompt with special instructions relevant to the functionality the RAG is expected to implement. The user input is then transformed into a database query, to query domain-specific knowledge - the Retrieval part, before feeding the retrieved data back to the LLM as context to the initial prompt (the Generation part) and thus generate the overall response. Prompt injection attacks occur in the initial stages of the RAG workflow and target the chatbot’s prompt template itself. The attacker can easily add some instructions to their prompt to replace the original template’s instructions with their own.

Some techniques exist to alleviate this issue, such as prompt filtering using Allowed/Disallowed lists of terms, or writing better templates; nevertheless, none of these prompt protection schemes can protect all prompts with absolute certainty. This is partly because GenAI does suffer, still, from hallucinations, and can therefore output completely wrong or unexpected responses.

Retrieval Protection

The next element after prompting is data retrieval, and this is really the area where we can enforce the best security. The retrieval part is a data protection problem, which is a well-known problem that has various solutions already implemented by various Access Management tools and systems. Dynamic access control, when applied to RAGs, must nevertheless implement slightly different features and configurations. The following rules should be implemented by any RAG protection scheme:

• No Standing Privileges. A RAG application or system is built to respond to user prompts. In particular, and this is key, when not prompted, the RAG system should do nothing at all (we are discussing RAGs here, not autonomous agents). This means that by default, a RAG system should have no access rights. If for some reason such a RAG system “decided” to query data of its own accord, it should have no access at all.
• Access Delegation. A user prompting a RAG application in-effect delegates their access rights and entitlements to the RAG for the duration of the prompt. Access delegation is a solved problem, for which ready solutions exist. RAG implementers can thus implement any existing scheme for user to RAG access delegation (for example, we can cite the OAUth2 Token Exchange standard as a possible solution). In any case, once the delegation is done, the RAG can use the token it has been granted to access the data store and retrieve the domain-specific knowledge it needs using the entitlements granted by that token. This ensures that the prompter will only ever see data they are entitled to see.
• Deterministic Dynamic Authorization. Access to the data should be controlled by a Policy Enforcement Point (PEP), a proxy service that can filter all requests and redirect them to an authorization service (a Policy Decision Point, or PDP), for evaluation. The important aspect of this PDP is that it must encapsulate deterministic access policies, i.e., rules that can be proven and produce predictable outputs. This PDP system must therefore not itself be a GenAI or LLM system, or else it may itself suffer from the same problems it is trying to alleviate.
• Knowledge-Based Access Control (KBAC). Traditional authorization models like ACLs, RBAC, and ABAC struggle with AI-driven systems that rely on knowledge graphs. These graphs, structured as nodes and relationships, add a semantic layer to data, often using embeddings—numerical AI-generated representations. KBAC aligns access control with this structure, leveraging contextual relationships and ontology-based policies to make informed authorization decisions. Unlike conventional schemes, KBAC enables security mechanisms that adapt to the semantic nature of RAG systems, ensuring more precise and effective access control.

GenAI protection is a nascent field, but proper controls are possible for responsible AI implementations. Organizations should look into enforcing at the very least the four rules detailed above as a basic safety net that can limit their RAGs security exposure.

Learn more about securing your AI deployments here.

^{This article was first published for RSAC™ Conference on February 10, 2024.}