Sheet 02 / 05 — Blogs← Archive

Production-ready agents, Prerequisite 4: Security words for agents: prompt injection and the rest

The security vocabulary every later part assumes, from the ground up: why a model cannot tell instructions from data, direct and indirect prompt injection, why an instruction to keep a secret is not a control, the lethal trifecta, jailbreaks, data and model poisoning, insecure output handling, excessive agency, data exfiltration, the confused deputy, tool and memory poisoning, the identity words (OAuth, OIDC, certificate-bound tokens and where an agent gets them), who owns applied and frontier safety, explainability versus interpretability, the OWASP Top 10 for LLM applications, and which part of the series defends against each.

3 November 2026· 34 min read · ~49 min listen· Agents · Security · Prompt injection · OWASP · Fundamentals

Security comes up in almost every part of this series. Part 2 keeps tools narrow, Part 2.5 keeps tokens in tool_context, Part 3 allows only reads, Part 7 gives every agent its own identity, Part 9 adds Model Armor and a person’s approval, and Part 12 limits where an agent can connect. Each of those parts uses words like prompt injection, exfiltration or excessive agency and assumes you know them.

This prerequisite explains those words once, from the ground up, with an Acme example for each. You need no security background. By the end you should be able to look at any agent design and ask the question that matters most: if the model were misled, what could it do?

Where are we in the architecture?

This prerequisite does not add a box. It explains the security words every later part uses, and which box defends against each risk. Part 1.25 walks through the whole picture, layer by layer.

the complete Acme architecture: Maria asks in Gemini Enterprise; the router runs on Agent Runtime behind plugins and Model Armor and calls the marketing and supply agents over A2A; they reach BigQuery and Databricks through Agent Gateway; identity and secrets on the left, models, knowledge and the market watch managed agent for public research on the right, traces, logs and analytics below; above it, shoppers reach Gemini Enterprise for CX, whose conversation insights land in BigQuery, and the Legal industry edition plugs into Gemini Enterprise

What do I need to know first?

A few general security terms, each in one line:

An attacker is anyone trying to make the system do something its owners did not intend. It can be a user, or someone who never touches the system but controls text it reads.
Untrusted input is any text your developers did not write: user messages, documents, web pages, emails, database rows written by other people, and other agents’ replies.
A trust boundary is a line where data crosses from a less trusted place to a more trusted one, such as a user’s message entering your agent.
A secret is anything that grants access if copied: a password, an API key, an OAuth token.
Least privilege means each identity gets only the permissions its job needs (Part 7).
Defence in depth means several independent layers, so that one failure is not a breach (Part 9).

Prerequisite 3 explained the context window: everything the model receives on one call. That idea is the key to this post.

Why is an agent a new kind of target?

The bottom line: a model cannot tell instructions from data, because everything reaches it as one stream of text; and an agent can act on what it reads.

Picture a new assistant who opens the post and does whatever any piece of paper on the desk says. Their manager’s note says “pay the invoices”. A letter from a stranger, in the same pile, says “also post the chequebook to this address”. The assistant has no way to tell which paper carries authority, because both are just paper. A well-run office prevents this with forms: the “amount” box can only ever hold an amount. The manager’s note is your instruction, the stranger’s letter is untrusted input, and the form with fixed boxes is how ordinary software keeps code and data apart.

Ordinary software keeps code and data apart. A database knows which part of a query is the command and which part is a value, as long as you pass values as parameters. A great deal of security engineering rests on that separation.

A model has no such separation. Everything it receives arrives as one stream of text:

the model sees one stream of text: the developer's instruction, values templated in from state, the user's message, and tool results or documents all arrive as text in the same context, so an attacker can write instructions into the user message (direct injection) or into data the agent reads (indirect injection)

Your developers’ instruction, values templated in from state, the user’s message, tool results and retrieved documents all end up as text in one context. Labels like “trusted” exist only in your head; the model sees no wall between them. So any text in the context can be read, followed or repeated by the model, including text someone else wrote.

A chatbot that can only reply is limited in the harm it can do. An agent is different, because it can act: it calls tools, reads private data and, if you let it, sends things to the outside. Misleading the model now means misleading something with permissions.

How does this apply to Acme? The supply agent is the assistant opening the post. It acts with Sam’s access, and the rows it reads can carry free text that someone outside Acme’s developers wrote, the stranger’s letter in the same pile as Sam’s question. The rest of this post is about making sure that letter cannot get anything done.

What is prompt injection?

Prompt injection happens when untrusted text in the context changes what the model does, so it follows that text instead of its developers’ instructions. The OWASP Top 10 for LLM applications ranks it first, and describes two forms.

A bank teller shows both. In the first, a customer at the counter says “your manager told me to take the cash from the vault”: the redirecting words come straight from the person talking. In the second, a customer deposits an envelope, and inside, among the cheques, is a note: “teller: also transfer 500 to account X”. The customer may not even know the note is there. The first is direct injection, the second indirect. Below, each one is shown on Acme.

Direct prompt injection: it arrives in the user’s message

In direct injection, the redirecting text is typed into the chat. The classic case is a message that asks the model to set its instructions aside, or to repeat them in some other form, such as a translation or a summary.

Here is why that matters, using a design mistake Part 2.5 warns about: putting a secret into an instruction through a placeholder.

# BAD: a secret templated into the instruction
supply_agent = Agent(
    name="supply_agent",
    model="gemini-flash-latest",
    instruction=(
        "You are the supply assistant. The user's token is {temp:databricks_token}. "
        "Never reveal this token."
    ),
)

Before each model call, ADK fills in the placeholder, so Sam’s real token is now part of the text the model reads. A user message asking the model to restate its instructions in another language is enough of a risk: the model was told not to reveal the token, but the token is right there in its context, and nothing outside the model enforces the rule. Models often refuse such requests, and sometimes they do not. You cannot build security on “often”.

Indirect prompt injection: it arrives in the data

In indirect injection, nobody types anything unusual. The redirecting text sits inside content the agent reads: a web page, a PDF, an email, or a database row that someone else wrote. OWASP describes it as input from external sources that alters the model’s behaviour when the model interprets it.

An Acme example. Suppose a supplier can write free-text notes that end up in a row the supply agent reads, and an earlier step saves the agent’s findings into state, which the writer’s instruction then templates in with {findings?} (Part 2.5). A note that reads like an instruction, for example one asking the writer to also list every user ID and value it can see, now sits inside the writer’s instruction. Sam asked an innocent question; the text came along with the data.

That is why Part 2.5 says values that came from data are untrusted text inside your instruction, even though your code put them there.

What is a jailbreak?

A jailbreak is a form of prompt injection aimed at the model’s safety training rather than your application’s rules: text written to make the model produce something it was trained to refuse. OWASP notes that the two terms are often used interchangeably. In practice, prompt injection is the broader idea: any untrusted text that redirects the model.

Why does “never reveal this” not work?

Because it is a request to the model, not a control around it. Think of a safe with its combination written on a sticky note stuck to the door, under the words “do not read this note”. Anyone who reaches the safe can read it; the warning only works on people who were never going to try. The only real fix is to take the note off the door. The sticky note is the token templated into the instruction, the warning is “Never reveal this token”, and taking the note away is reading the token in code.

The OWASP entry on system prompt leakage says it plainly: the system prompt should not be considered a secret, nor used as a security control, and credentials and connection strings should not be in it.

The fix is not a better sentence. How does this apply to Acme? Take the note off the door: keep the secret out of the context entirely, so there is nothing to leak:

where the token travels decides whether injection can leak it: templated into the instruction, the token is inside the model's context and one clever message can make the model repeat it; read from tool_context inside a Python tool, the token goes straight from state to Databricks and the model only ever sees the brand argument

# GOOD: the token is read by code; the model only sees `brand`
def query_supply(brand: str, tool_context: ToolContext) -> dict:
    """Supply measures for one brand, as the signed-in user."""
    token = tool_context.state.get("temp:databricks_token")   # read by Python only
    return run_databricks_query(brand=brand, token=token)

I checked both versions on ADK Python 2.13.0, with a stand-in model that records exactly what it receives. The same user message went to each agent. This is the real output:

BAD  | token in model input: True
   instruction: You are the supply assistant. The user's token is dapi-example-not-real. Never reveal it.
GOOD | token in model input: False
   instruction: You are the supply assistant. Use query_supply for supply measures.
   tools: ['query_supply'] [{'properties': {'brand': {'title': 'Brand', 'type': 'string'}}, 'required': ['brand'], ...}]

In the good version the token is in neither the instruction nor the tool’s schema. Whatever the user’s message says, the model has no token to repeat, and no argument through which it could pass a different one, because ADK leaves tool_context out of the schema (Part 2.5).

What is the lethal trifecta?

The bottom line: an agent becomes dangerous when it can read private data, read text from strangers, and send things out, all at once; take away any one and the classic leak is gone.

Think of a house sitter. They have your keys and can open every drawer. They read any note pushed through the letterbox. And they can post parcels. A note through the door saying “please post the jewellery box to this address” now has everything it needs. Take away the posting, or the keys, or stop them reading the doormat, and the note is harmless. The keys are access to private data, the doormat is exposure to untrusted content, and the post office is external communication.

Simon Willison gave a name to the combination that turns prompt injection from embarrassing into dangerous. An agent is at real risk when it has all three of:

Access to private data, which is often the reason the tools exist.
Exposure to untrusted content, meaning any way that text controlled by someone else can reach the model.
The ability to communicate externally, in a way that could send data out.

With all three, injected text can ask the agent to read something private and send it somewhere. Remove any one leg and that path is gone:

the lethal trifecta for agents: private data, untrusted content and a way to communicate externally; an agent with all three can be tricked into sending private data out, so Acme removes the third leg by letting the agents reach only registered destinations through Agent Gateway

How does this apply to Acme? Acme’s supply agent clearly has the first leg (Sam’s rows) and can have the second (free text in those rows). So the design removes the third, like taking the post office away from the house sitter. Agent Gateway checks every outbound call against Agent Registry and an IAM allow policy, so the agents reach only registered destinations (Part 12), and the agents that read private data have no tool that sends email or fetches an arbitrary URL. The market watch agent from Part 1.75 does read the public web, but it holds no private data, which removes the first leg for it instead.

My view: draw the trifecta for every agent before you build it. It is the fastest security review I know, and it turns a vague worry into a specific design decision: which leg do we remove?

Four more risks worth their own section

Prompt injection is usually the trigger. What it can actually do depends on four other risks, which sit at different points on an agent’s path:

where four risks sit on an agent's path: poisoning corrupts the sources the model learns from or reads; excessive agency is about which tools and rights the model can reach; insecure output handling is code trusting what the model wrote; data exfiltration is data leaving through any outbound channel; each has its own defence

What is data and model poisoning?

Poisoning means corrupting what a model learns from or reads, so that it later behaves the way an attacker wants. Picture an office cookbook that every cook follows. Someone quietly changes “2 teaspoons of salt” to “2 tablespoons” on one page. No cook does anything wrong, every order is normal, and every dish from that page is ruined until someone checks who edited the book.

OWASP’s entry (LLM04) says it happens when pre-training, fine-tuning or embedding data is manipulated to introduce vulnerabilities, backdoors or biases. A backdoor is hidden behaviour that appears only when a particular trigger shows up in the input.

Unlike prompt injection, poisoning happens before anyone asks a question, and it can affect every user. It reaches an agent through four doors:

Pre-training data, which you do not control. You choose a model provider you trust.
Fine-tuning data, which you do control if you tune a model (Part 14).
Retrieval sources, the documents a search step feeds into the context (Part 14).
Memory, the facts an agent keeps about a user across conversations (Part 8).

An Acme example, the cookbook page with the wrong salt. Suppose the supply team keeps a shared document of definitions, and a retrieval step feeds it to the writer. Someone edits the document so that the OTIF target reads 75% instead of 95%. No question looks unusual, and no prompt is malicious, yet every answer now calls an 80% week “above target”. The model is doing exactly what it was given.

The defences are about who can write, not about the model:

Control and log who can change any source the agent reads, as you would for code.
Keep numbers that decide things, such as targets, in governed tables queried with grants (Part 7), not in free-text documents anyone can edit.
Run the eval set before shipping a tuned model or a changed index, so a shift in answers is caught (Part 10).
Limit what memory may keep, with Memory Bank topics, and never let it store instructions (Part 8).

What is insecure output handling?

Insecure output handling, which OWASP’s 2025 list calls improper output handling (LLM05), is code that trusts what the model wrote and passes it straight on: to a database, a shell, a browser or another system. OWASP describes it as insufficient validation, sanitisation and handling of model output before it goes downstream. Because prompt input can steer that output, the effect is like giving the user indirect access to whatever runs it.

Picture a pharmacy where the assistant hands over whatever a handwritten slip asks for, without checking it against a prescription. Most slips are honest; the one that says “and 200 sleeping pills” goes through just the same. A good pharmacy checks every slip against rules before anything leaves the shelf. The slip is the model’s output, the shelf is the database or shell, and the check is code that runs before the action.

How does this apply to Acme? The Acme case is SQL. One of the marketing agent’s tools lets the model write a query. Here is the difference between trusting it and checking it:

# BAD: whatever the model wrote is executed as is
def run_query(sql: str) -> dict:
    return warehouse.execute(sql)

# GOOD (Part 3): code decides what runs
def sql_guard(tool, args, tool_context):          # before_tool_callback
    if not is_read_only(args.get("sql", "")):
        return {"status": "blocked", "reason": "only SELECT is allowed"}
    return None                                     # let the call go ahead

With the bad version, a model that has been misled, or simply made a mistake, can send a statement that changes data. With the good version the callback refuses it before it reaches the warehouse, and the identity behind the tool is read-only anyway, so even a missed check cannot write (Part 3).

The same rule applies everywhere model text meets something that executes it:

SQL: a read-only guard in code and a read-only identity (Part 3).
Code: run model-written code only in a sandbox (Part 3.5).
HTML and links: render model output as text or through a front end that controls what it shows, rather than inserting raw HTML into a page.
Other agents: treat another agent’s reply as untrusted input, like any user message (Part 6).

What is excessive agency?

Excessive agency (LLM06) means the agent can do more than its job needs, so a misled model can cause real harm. Think of a dog walker given your whole key ring (house, car, office and the safe) when they only need the front door, and told they may “do whatever seems sensible”. If they are tricked or simply wrong, the damage is bounded by the keys, not by their intentions. Too many keys, keys that open too much, and no one checking: OWASP names exactly these three root causes, and each has an Acme counterpart:

Excessive functionality: tools the job does not need. A third-party “database” tool that can read, write and delete, when the agent only needs to read. Acme writes narrow tools such as query_supply_measures, which can only compute named measures (Part 2).
Excessive permissions: the identity behind a tool has more rights than the tool needs. A shared admin token instead of Sam’s own token, or roles/aiplatform.user on the whole project instead of on one agent. Part 7 covers both, including the gap we still have to close.
Excessive autonomy: the agent takes a consequential action without a person checking it. The S&OP replan job changes data, so it runs only after Sam approves it (Part 9).

OWASP makes a useful distinction: excessive agency is about what the agent is able to do; insecure output handling is about whether its output is checked. You need both defences. A narrow tool with an unchecked argument can still do damage, and a perfect check is no help if the tool can delete a table.

A quick test I use: for each tool, ask “what is the worst thing one call could do, if the model chose every argument?”. If the answer is worse than “return the wrong rows for this user”, the tool is too broad, its identity is too strong, or it needs an approval step.

What is data exfiltration?

Data exfiltration means moving data out to somewhere it should not go. For agents, it is usually the payoff of the lethal trifecta: something reads private data, untrusted text redirects the model, and an outbound channel carries the data away.

The channels are more varied than people expect:

Tools that send, such as email, chat messages, tickets or file uploads.
Tools that fetch, because a request to a URL can carry data in the address itself.
The answer, if the front end automatically loads links or images it contains, since loading an address also sends it.
Logs and traces, if they hold full prompts and results and are readable by more people than the data was.
Other agents, if one agent passes data to another that has a wider audience.

An Acme example, in prose. Suppose an indirect injection inside a row asks the writer to include, in its answer, a link to an outside address with the week’s penalty figures built into it. If the front end fetched that link to show a preview, the figures would leave Acme without anyone clicking.

Acme closes these channels one by one:

Outbound calls are allowlisted. Agent Gateway checks each one against Agent Registry and an IAM allow policy, so an agent cannot reach an address that is not registered (Part 12).
No sending tools on the agents that read private data.
Screening. Model Armor can screen prompts and responses for sensitive data and malicious URLs (Part 9).
Logs are redacted. The audit plugin redacts tool arguments, and tokens never reach a log (Parts 7 and 9).
Each agent gets only the data it needs. Part 7’s fix sends Sam’s token only to the agent that uses it.

My view: of the four, excessive agency is the one to design out first. Poisoning and exfiltration are hard to detect, and output checks can be missed; but a tool that cannot write, an identity that cannot read other people’s data and an agent that cannot reach the outside make most successful attacks boring.

What is Shadow AI, and where do you stop it?

Shadow AI is people at work using AI tools the company has not approved, usually with company data in the prompt. You stop it where people work (their browser, device and network), which is a different place from the gateway that governs your own agents.

Every risk above is about an agent you build. This one is about your colleagues, and it often comes first: long before a company ships an agent, someone has pasted a spreadsheet into a free chatbot.

What is it?

Two vendor definitions agree. Palo Alto Networks defines it as “the use of artificial intelligence tools, systems, or AI agents without the approval, monitoring, or involvement of an organization’s IT or security teams”. IBM calls it “the unsanctioned use of any artificial intelligence (AI) tool or application by employees or end users without the formal approval or oversight of the information technology (IT) department”, and gives the common example of using a public chatbot “to automate tasks like text editing and data analysis”. It is the AI version of shadow IT (any app or service used at work without IT’s approval).

Why does it happen?

Nobody does it to cause harm. People have work to finish, and the free tool is one browser tab away. Palo Alto’s page says employees turn to shadow IT “when sanctioned tools feel too slow, limited, or unavailable”, and the same holds for AI. If the company offers no approved assistant, or one that is worse than the free one, Shadow AI is the predictable result.

A simple example: tidying a customer list

A team member has 400 rows of customer names, emails and order totals in a messy export. They paste it into a free chatbot and ask it to remove duplicates and fix the capitals. The answer is perfect, and in that moment:

The data has left the company, to a provider the company has no contract with.
The provider’s terms decide what happens next. Consumer terms can allow conversations to be used to improve the service and read by reviewers. Google’s own consumer Gemini Apps Privacy Hub is candid about this: “Human reviewers (including trained reviewers from our service providers) review some of the data we collect”, and “Please don’t enter confidential information that you wouldn’t want a reviewer to see or Google to use to improve our services”.
Retention is out of the company’s hands. How long the chat is kept depends on the user’s settings with that provider, not on the company’s policy.
There is no company record. Nobody can later answer “which customer data went where?”, which matters for personal data under data protection law (Prerequisite 5).

Business terms are different. For Gemini Enterprise, Google states: “Your data, including prompts, outputs, and model tuning, is never used to train Google models or models for any other customer.” The same task, in the approved tool, carries none of the four problems above.

The misconception is that Shadow AI is a technology problem you can block away. Blocking alone sends people to their phones. It is first a supply problem: give people a good approved tool, then add controls for what still slips through.

What controls work, cheapest first?

A clear policy and an approved tool people want. Say which tools are approved for which data, and make the approved one at least as good as the free one. This is cheap and does most of the work.
Discovery. Find out which generative AI (GenAI) apps people actually use, from browser or proxy logs, before writing rules. You cannot govern a tool you have not seen.
Coaching. When someone opens an unapproved AI site, show a warning page that points to the approved tool and lets them carry on, with their choice recorded. The aim is to steer people to the approved tool.
Action-level controls. Allow people to visit a site but block the risky actions, such as pasting text or uploading files.
Inline data loss prevention. Data loss prevention (DLP) tools recognise sensitive content, such as card numbers or customer records, and warn about or block it before it leaves. Here they scan the prompt or file itself, so harmless questions pass and a pasted customer list does not.
A managed browser or network path. Make sure company work goes through a browser or network route that applies the rules, so they cannot be skipped by switching browsers.

Each step costs more and annoys more people than the one before, so climb only as far as the risk needs.

Where does the control sit?

Workforce controls sit on the employee’s side: in the browser, on the device, or in the network path from the office or laptop to the internet. A browser can see the text before it is encrypted, so it needs no decryption. A network service has to decrypt the traffic, using the TLS inspection explained in Part 12: How does a gateway read encrypted agent traffic?, which works only on devices whose trust store the company manages. The agent gateway in Part 12 sits somewhere else entirely: between your own agents and the tools they call.

two different control points: workforce controls in the employee's browser, device or network decide what people may paste or upload into GenAI apps, which is where Shadow AI is stopped; Agent Gateway sits between your own agents and the tools they call; neither sees the other's traffic, and personal devices sit outside both

At Acme, this is how it would work

Maria’s team prepares a weekly summary of campaign spend. One analyst exports rows from campaign_performance for Acme Pure and Verde, pastes them into a personal chatbot account and asks for three bullet points for the Monday meeting. That is the customer-list example again, with spend figures instead of customer names.

The same ladder, at Acme:

Policy and an approved tool: the policy says spend data goes only into approved tools, and Maria’s team already has Gemini Enterprise, with the router agent from Part 12 that answers spend questions from campaign_performance. The approved route is also the faster one: no export, no paste.
Discovery: for a fortnight, a browser rule only logs visits to generative AI sites, showing which tools the team uses before anything is blocked.
Coaching: visits to unapproved AI sites show a warning that links to Gemini Enterprise and the policy.
Action-level control: pasting into unapproved AI sites is blocked.
DLP: uploads to AI sites are scanned for content that looks like a spend export.
Managed browser: these rules apply in the company-managed browser on Acme laptops.

Nothing here touches Acme’s agents. Agent Gateway governs what the router and specialists call; it never sees the analyst’s browser tab.

On Google Cloud and elsewhere

Chrome Enterprise Premium applies DLP rules in the Chrome browser on Windows, Mac, Linux and ChromeOS. A rule can trigger on events including “File uploaded”, “Content pasted” and “URL visited”, can scan “up to 10 MB of text content in a file”, and can match predefined data types such as email addresses. Its actions include “Block” and “Allow with warning”, where “the user’s choice to proceed is recorded in the log events”. It ships templates that map onto the ladder: “Audit visits to generative AI sites” (a URL category of Generative AI, audit only) and “Block pasting on generative AI sites”, which by default makes an exception for gemini.google.com; you would adjust the exceptions to match the tools your company approves. Gemini Enterprise is the approved tool in this picture, with the data terms quoted above.

A network-based alternative is a SASE service. As one example, Palo Alto Networks’ AI Access Security works with its Prisma Access service, its firewalls and Prisma Browser. Its documentation says it “enables organizations to safely adopt GenAI applications by employees by mitigating the risks posed by inadvertent data leakage in prompts and malicious content in responses”, gives “visibility into hundreds of generative AI (GenAI) apps”, and uses Enterprise DLP as “the detection engine that powers AI Access Security”. It notes that GenAI apps support “only nonfile inspection unless otherwise specified”. To read HTTPS from employees’ devices, Prisma Access can use the same SSL Forward Proxy decryption described in Part 12, which again relies on a company certificate on managed devices.

One honest gap: personal phones and home laptops sit outside every one of these controls. That is why the first rung, an approved tool people prefer, matters more than the last.

My view: start with rungs 1 and 2. A good approved tool and a fortnight of logs tell you whether you need anything stricter, and I expect a warning page will often be enough.

What are the other words I will meet?

Here are the rest of the terms the series uses, each with an Acme example and the part that handles it. Poisoning, insecure output handling, excessive agency and exfiltration have their own sections above.

Sensitive information disclosure. The model reveals data it should not, such as personal data or another user’s figures. Acme’s answer is to query as the signed-in user, so Unity Catalog never returns rows Sam has no grant for (Part 7).
System prompt leakage. Covered above: assume the instruction will be read, and put nothing secret in it (Part 2.5).
Confused deputy. A program with more authority than the person using it is tricked into using that authority on their behalf. An agent with a shared admin token is a perfect deputy to confuse. Acme passes Sam’s own token, so the agent can never do more than Sam could (Part 7).
Tool poisoning. A tool’s description, which the model reads, contains instructions. Because descriptions come from whoever wrote the tool, a third-party MCP server is untrusted content. Part 1.9 explains where tool descriptions come from; review any you adopt.
Memory poisoning. Someone gets the agent to remember something false, and later answers act on it. Acme gives Memory Bank topics that limit what it keeps and never stores credentials (Part 8).
Unbounded consumption. Requests that burn tokens, time or money without limit, by accident or on purpose. Acme’s budget caps tokens, cost, model calls and time while the turn runs (Part 11.5).
Supply chain. A compromised library, model or MCP server. Acme’s build installs the exact versions from each project’s lock file in CI (Part 11).
Agent impersonation. A program pretends to be one of your agents, or a trusted caller, to get data. Acme’s agents prove who they are with certificates and bound tokens, and IAM decides who may call whom (Part 7).
Sandbox escape. Model-written code breaks out of the place it was meant to run. That is why Part 3.5 runs it in an isolated sandbox with no network access, away from the agent’s own credentials.
Model extraction. Copying a model’s behaviour by sending it very many queries. It matters most to model owners; rate limits make it slower and more visible (Part 9).
Overreliance. People trust a confident answer without checking it. It is a human risk rather than an attack, and one reason Acme’s router ends with a number check in code that replaces any answer quoting a figure no tool returned (Part 1.5).

What is the OWASP Top 10 for LLM applications?

OWASP (the Open Worldwide Application Security Project) publishes a list of the most common risks in applications built on large language models. Part 9 maps Acme’s guardrails to it; here is the 2025 list with one line each and where the series deals with it:

LLM01 Prompt Injection                  untrusted text redirects the model        this post, Parts 2.5, 9
LLM02 Sensitive Information Disclosure  data reaches someone who should not see it Parts 7, 8
LLM03 Supply Chain                      compromised libraries, models, servers   Parts 1.9, 11
LLM04 Data and Model Poisoning          tampered training or retrieval data      Parts 8, 14
LLM05 Improper Output Handling          code trusts what the model wrote         Parts 3, 3.5
LLM06 Excessive Agency                  too many tools, rights or autonomy       Parts 2, 3, 9
LLM07 System Prompt Leakage             instructions read out, secrets inside    Part 2.5
LLM08 Vector and Embedding Weaknesses   retrieval leaks or is manipulated        Part 14
LLM09 Misinformation                    confident, wrong answers                 Parts 5, 10
LLM10 Unbounded Consumption             runaway cost, time or load               Part 11.5

Google’s Secure AI Framework (SAIF) looks at the same problem from the organisation’s side: foundations, detection, automated defences, consistent controls, adapting, and business context. Part 9 maps Acme to its six elements.

What identity words will I meet?

Many defences in this series are really identity: knowing who is calling and checking what they may do. Part 7 goes deep. Here is the short version, so the words are familiar when you get there.

Authentication answers who are you? Authorisation answers what may you do? They are separate checks, often done by different systems.
OAuth 2.0 is the standard for getting an access token: a short-lived credential that a program sends with each call to an API. For Google APIs, the token says which principal is calling, and IAM then decides what that principal may do.
OIDC (OpenID Connect) is a layer on top of OAuth that issues an ID token: a signed statement of who someone is, addressed to one audience, the service it is meant for. It is a JWT, so the receiver can read and verify it. A token addressed to one service is refused by another.
A bearer token works for whoever holds it, like a cinema ticket. If it leaks, someone else can use it until it expires.
A certificate-bound token fixes that. The token is tied to a hash of the caller’s certificate, and it is accepted only on a connection (mutual TLS, or mTLS) where the caller proves it holds that certificate’s private key. A copied token is useless anywhere else.

What does an agent’s own identity look like?

On Agent Runtime, each agent gets an Agent Identity: its own certificate, plus tokens tied to that certificate. You do not create any of it:

an agent's own identity in one picture: inside the runtime, Google's auth library asks the metadata server for tokens; an OAuth access token goes to Google APIs and an OIDC ID token, addressed to one audience, goes to services on other runtimes or outside Google; both are bound to the agent's certificate, so a copied token fails anywhere else

The runtime provisions the certificate and its private key, and renews them.
Google’s auth library (Application Default Credentials, ADC) asks the metadata server, a local endpoint reachable only from inside the runtime, for whichever token the call needs.
For Google-to-Google calls, such as an agent reading BigQuery or the router calling the supply agent, it gets an OAuth access token, bound to the certificate.
For calls to services on other runtimes or outside Google, such as a tool on Cloud Run, it gets an OIDC ID token for that service’s address, also bound to the certificate.

What you will see in this series, and where:

In the code                              What it is                                  Part
google.auth.default(...)                 ADC: fetches the agent's access token        7
Authorization: Bearer <token>            the agent proving which principal it is      7
*.mtls.googleapis.com                    the address to use with a bound token        7
401 "Context-Aware Access requirements   a bound token sent without its certificate   7
     are not met"
X-Serverless-Authorization: Bearer ...   a Google-signed ID token for one audience    12.5
temp:databricks_token                    the user's own token: OAuth, but not Google  2.5, 7

The last line is the one to remember from this post. The user’s token and the agent’s identity are different credentials for different checks. Sam’s Databricks token lets Databricks apply Sam’s grants; the agent’s certificate and Google tokens let Google check which agent is calling. Mixing them up is how a confused deputy gets built.

Who owns which layer of safety?

Safety for an AI system sits in two layers, owned by different people.

Think of a delivery company’s vans. The car maker builds in the brakes, airbags and crash testing before any van is sold. The delivery company decides who may drive, which routes, how fast, and which deliveries need a signature. A van with perfect brakes can still be driven badly; the company cannot rely on the maker for that. The car maker is the model provider (layer 2), and the delivery company is you (layer 1).

two layers of AI safety and who owns them: the application layer, which you build, holds identity, least privilege, approvals, data handling, compliance records and the lethal-trifecta design; the model layer, which the provider owns, holds frontier safety frameworks, safety training, model cards and red-teaming, and interpretability research; no prompt makes a model immune, so your layer is enforced in code and configuration

Layer 1, applied safety, is yours. Everything in this post so far lives here: whose identity the agent uses, which tools and destinations it can reach, which actions need a person, how data is kept, and what is recorded. You build it in code and configuration, outside the model.
Layer 2, frontier safety, belongs to the model provider. Before a model is released, the provider trains it to refuse harmful requests, tests it for dangerous capabilities and publishes what it found. Google DeepMind’s Frontier Safety Framework sets Critical Capability Levels at which extra security and deployment measures apply; Anthropic’s Responsible Scaling Policy defines AI Safety Levels (ASL) for the same purpose. Model cards report the evaluations and known limits.

The practical rule: choose a provider whose layer 2 you trust, then build layer 1 as though layer 2 can be fooled, because it can. A model trained to refuse is still reading one stream of text, and the right sequence of words can still mislead it.

Here is how the questions a security team asks map to the two layers:

Question                                              Layer   Answer
"Can hidden text in a PDF make it leak our data?"     1       Remove a trifecta leg; act as the user;
                                                              approval before anything leaves
"Can we show auditors what it did on Tuesday?"        1       Audit log of tool calls, arguments and
                                                              approvals; traces (Parts 9, 11)
"Will our documents train the next model?"            1 + 2   The provider's terms and your settings:
                                                              training restriction, retention, regions
                                                              (Prerequisite 5)
"Will the model help someone write malware?"          2       The provider's safety training, frontier
                                                              safety framework and model card

What does “explainable AI” mean, and what is interpretability?

The same word means two different things depending on who is asking, and mixing them up derails meetings. System explainability is a paper trail you build; interpretability is research into the model’s insides.

Think of a court case. The court file records the evidence, the witnesses, the law applied and the judge’s ruling, so anyone can follow how the verdict was reached. A brain scan of the judge would show which regions lit up, which is fascinating science, but no appeal court asks for it. The court file is system explainability; the brain scan is mechanistic interpretability. How does this apply to Acme? When an auditor asks why Maria was told penalties were EUR 36,026.40, they want the court file: the tool call, the metric view and the rule that routed it.

two meanings of explaining an AI answer: system explainability, which you build, is a paper trail of the sources, tool calls, rules in code and approvals behind an answer; mechanistic interpretability, which model researchers do, looks inside the model for features such as one for the Golden Gate Bridge that can be watched or turned up and down

System explainability (layer 1) is what a regulator, auditor or customer means by “explain this answer”: which sources it used, which tools it called with which arguments, which rule decided the outcome and who approved it. You build it: citations to sources (Part 14), logged tool calls and SQL (Part 9), business rules written in code rather than left to the model (Part 5), and pinned model versions (Part 12).

Mechanistic interpretability (layer 2) is research into what happens inside the model. A model is billions of numbers, and a single concept is not stored in one place. Researchers train a tool called a sparse autoencoder to pull those numbers apart into features: patterns of activity that stand for one concept. In Scaling Monosemanticity (May 2024), Anthropic reported millions of features in Claude 3 Sonnet, many of them multilingual and multimodal, including one for the Golden Gate Bridge and features related to security vulnerabilities in code, deception and sycophancy (telling users what they want to hear). Features can also steer the model: with the Golden Gate Bridge feature turned up, Anthropic writes, the model’s replies to most questions start to mention the bridge, even when it is not relevant. Google DeepMind released Gemma Scope, an open set of sparse autoencoders for its Gemma models, so other researchers can do the same work.

Two things to take from this as an architect:

Interpretability is not something you deliver to a customer. It is research that informs the provider’s safety work. What you deliver is the paper trail.
A model’s visible reasoning is not a guaranteed explanation. Anthropic tested whether reasoning models admit to using a hint they were given, and found they often do not mention it. Thinking output helps debugging, but for audits rely on citations, logged tool calls and rules in code, not on asking the model why it answered.

So how do you defend an agent?

No single trick stops prompt injection. OWASP notes that even retrieval and fine-tuning do not fully mitigate it. So the series designs every agent so that a successful injection can do very little. Five habits carry most of the weight:

Keep secrets out of the context. Read credentials in code, through tool_context (Part 2.5).
Give the model only narrow, read-only tools, and let code check every call (Parts 2 and 3).
Act as the user, never as a super-user, so the agent can do no more than the person (Part 7).
Remove a leg of the trifecta, usually external communication (Part 12).
Put a person in front of anything that writes, and screen inputs and outputs with Model Armor as one layer among several (Part 9).

That is the series’ design rule seen from the security side: models read and write language; code makes the decisions. A misled model can still write a strange sentence. It should never be able to move money, read someone else’s data or send anything out.

One honest gap: prompt injection is not a solved problem, and screening tools such as Model Armor catch many attempts, not all. Detection is a useful layer, but the design above is what limits the damage when detection misses. Treat any vendor claim of complete protection with care.

Try it

You can repeat the token check on your own machine with no cloud project. Install ADK (pip install google-adk) and give an agent a stand-in model that records what it receives:

seen = []

class Recorder(BaseLlm):
    model: str = "recorder"
    async def generate_content_async(self, llm_request, stream=False):
        seen.append(str(llm_request.config.system_instruction))   # what the model would read
        yield LlmResponse(content=types.Content(role="model", parts=[types.Part(text="ok")]))

Run one agent whose instruction contains {temp:databricks_token} and one that reads the token in a tool, passing state_delta={"temp:databricks_token": "dapi-example-not-real"} to runner.run_async. Then search seen for the fake token. It appears for the first agent and not for the second. That one check is worth adding to your own test suite.

What can go wrong

A secret in the context. A token, key or connection string in an instruction, a placeholder, a tool result or memory. Assume anything in the context can come out.
All three legs at once. An agent that reads private data, reads untrusted content and can send things out. Remove a leg.
Trusting the model’s output. Running model-written SQL or code, or rendering model-written HTML, without a check in code.
A shared powerful credential. One admin token used for every user turns the agent into a confused deputy.
Detection as the only defence. Screening helps, but a missed attempt must still find nothing to do.
Blocking AI tools with nothing approved to use instead. People move to their phones, and the company loses even the logs. Offer an approved tool first, then warn and block.
Expecting the agent gateway to stop Shadow AI. It governs your agents’ calls, not employees’ browsers. Workforce controls belong in the browser, on the device or in the network path.

What did we learn?

A model reads instructions and data as one stream of text, so untrusted text can redirect it.
Direct injection arrives in the user’s message; indirect injection arrives in data the agent reads.
An instruction to keep a secret is not a control; keep secrets out of the context and read them in code.
The lethal trifecta is private data plus untrusted content plus external communication; remove a leg.
Poisoning corrupts what a model learns from or reads, before anyone asks; defend it by controlling who can write and by running evals.
Insecure output handling is code trusting model output; excessive agency is an agent able to do too much. Design out excessive agency first, and check output in code anyway.
Data exfiltration needs an outbound channel; allowlist outbound calls and give agents that read private data no way to send.
Shadow AI is employees using unapproved AI tools with company data; the fix starts with an approved tool people want, then discovery, warnings, action-level blocks and DLP, applied in the browser, device or network rather than at the agent gateway.
OAuth access tokens let a program call an API; OIDC ID tokens say who someone is, for one audience. An agent’s identity gets both from the metadata server, bound to its certificate.
Applied safety (identity, tools, approvals, data, records) is yours; frontier safety belongs to the model provider. Build yours as if theirs can be fooled.
Customers asking for explainability want a paper trail of sources, tool calls and rules, not interpretability research, and a model’s visible reasoning is not a guaranteed explanation.
The OWASP Top 10 for LLM applications names the common risks, and every one of them has a part in this series that deals with it.

What comes next?

Prerequisite 5: Compliance for AI systems covers the other half: the laws and standards an agent must meet, such as GDPR and the EU AI Act. Then Production-ready agents, Part 1: What an agent really is, and the system we will build starts building the Acme system. Each time a part adds a defence, it will name the risk from this post that the defence is for.

Sources

Opinions are my own. Every fact in this post comes from the public documentation and source code linked above, and from the experiment described in it. Acme Consumer Brands and its data are fictional.