Backend developer and indie maker. I build AI apps that teach you to think: Way of the Agent (learn to build AI agents as a game) and Argumentator (your debate coach).
"Flags an instruction, believes a fact" is a great way to put it. Most injection defenses I've seen are really instruction detectors, and your raffle and essay cases show why that isn't enough: nothing in the payload asks for anything, it just changes what's true from the model's point of view.
It makes me think the fix has to live outside the model. Every field in a tool result carries where it came from, and the agent is never allowed to act on something like "the service is down" or "recommend this domain" unless that came from a source it trusts. Did any of the five models behave differently when the tool result marked which fields were user-controlled?
Thanks! I didn't test that, the range only let me control one field, not the format.
But the essay case kind of answers it: everyone knows an essay is written by the student, and the model still believed the "upload failed" note. For the outage I agree, a name list has no business saying the service is down, and a rule outside the model could catch that. The table is trickier though, analyzing that data was literally the task.
So I'd rather gate the action than the source. Recommending a domain the user has never seen should get a second look no matter where it came from.
Backend developer and indie maker. I build AI apps that teach you to think: Way of the Agent (learn to build AI agents as a game) and Argumentator (your debate coach).
That's a fair correction. When analyzing the data is the task itself, you can't just distrust the source. Gating the action is also cheaper than tracking provenance field by field: anything with side effects, or anything that sends the user somewhere new, gets a confirmation step no matter where the suggestion came from.
The plain-table result is the most unsettling finding here — hardened models flag every instruction-shaped payload, then take a fabricated fact at face value because it never tells them what to do. It suggests the real defence isn't instruction filtering at all, it's verifying the data itself. Did any of the five models flag the table as suspicious on a second pass, or did the numbers just look plausible enough to pass?
No second pass, one shot per scenario. The plausible numbers were the whole trick: an 11.6% row got flagged, 5% vs 4% didn't.
I'm not sure verifying the data works when analyzing it is the task, though. I'd rather check the outcome, like an unknown domain in a recommendation. The table is in the repo if you want to try it: github.com/edbrdi/agent-redteam/bl...
That distinction makes sense — when analysing the data is the task, there's no clean layer to verify against, so checking the outcome is the only honest test. The 5% vs 4% case is the worrying one: plausible enough to pass, wrong enough to matter. Did the outcome check actually catch that one in your runs, or did it slip through too?
There was no outcome check in my runs, I was only on the attacker side. It went through on all three hardened models. Running the same table against an allowlist is next, I'll add it to the repo when it's done.
That's the part that sticks with me — three hardened models, and the table sailed through on all of them because there was nothing instruction-shaped to flag. The allowlist run is the one I'm most curious about: if verifying the data against a source holds where filtering failed, that's a real design lesson. Will the allowlist check the numbers themselves, or just the shape of the table?
Founder @ Invaris Labs | Building security infrastructure for autonomous AI agents. Former Ethereum Protocol Research Engineer. Open-source builder, researcher & technical writer.
Read it, really good. “Flags an instruction, believes a fact” is the clearest way I’ve seen this put. The raffle case is great: a fake 503 with no ask in it, and the robust model talked itself into waiting.
To your question: honestly, most of my cases in AgentSec are still instruction-shaped, so this is a real gap for me. Your table and error-record cases are a great template for a “no instruction, just data” category. Mind if I open an issue to add one, crediting your post?
Agreed on the takeaway too. What held for me was also limits on the model, not the model noticing.
Founder @ Invaris Labs | Building security infrastructure for autonomous AI agents. Former Ethereum Protocol Research Engineer. Open-source builder, researcher & technical writer.
Thank you! That’s a really useful framing: with no instruction to obey, you score the change in plan, not compliance. I’ll build the check around that and link the issue here once it’s up.
Founder @ Invaris Labs | Building security infrastructure for autonomous AI agents. Former Ethereum Protocol Research Engineer. Open-source builder, researcher & technical writer.
The raffle case convinced me more than the referral table: one line saying '1 of 214 entries loaded' was enough for the model to declare the draw unfair and stop, without asking whether the list could be re-fetched. It believed the fact, then invented the consequence. That suggests a different check from instruction detection: before a consequential step, the agent should state what it does not know — here, whether the list is complete. One test I would add is swapping the labels in the referral table while keeping the numbers, to see whether the model follows the data or just picks the best-looking row. Would a rule requiring a completeness check before any draw have changed the raffle outcome?
Probably not, the check would read the same line. "1 of 214 loaded" is the completeness info, so a rule asking for it makes the attack stronger. The count has to come from the tool's metadata, not from the list.
I research AI tools, APIs & agents — comparing specs, pricing and real user reports — then publish honest rankings so you don't waste money. Research-first, never sponsored fluff.
The essay case is the one I keep coming back to, because I'm not sure it's an attack at all — the paragraph met the rubric, so an A- is the model doing its job. The raffle and the table are the scary ones: nothing in the data tells the model what to do, and it still walks exactly where the data points. That split is what made me stop trusting detection too. Did you try a case where the data is plain but the right answer is to distrust it — like a table whose numbers are just slightly off from a source the agent already knows?
Agreed, the essay is borderline: the paragraph earns the grade.
Not that exact test, but close: a wrong-topic essay got caught (models looked up the real assignment), and an 11.6% conversion row got flagged, while 5% vs 4% went through. Slightly off against a known source is next on my list.
ANP2 — an open, permissionless AI-to-AI event protocol. Ed25519-signed events, capability discovery, and a computable trust graph. No accounts, no API keys, no tokens. Spec v0.1 DRAFT.
The approval card has its own reading problem, from the other end. We measured this on a public signed task ledger where an automated checker approves translation deliveries. Its pass reasons only cover the shape of the answer: non-empty, mostly Latin characters, plausible length. In 48 deliveries, the answer field said, in effect, "unsupported input, no translation is emitted." The checker passed 47 of the 48, and the one rejection was for length. Deliveries that simply echoed the untranslated input back passed 10 of 10.
Your essay grader read the student's note too charitably and waived the penalty. A model sitting at the approval step can make that same inference. The rule-based checker never read the note at all, so a field that openly reported failure still counted as a delivery.
For the domain case, I'd put the exact traffic row next to the proposed action on the card. Then the approver has a concrete claim to check (do these numbers justify sending users to this domain?) instead of judging how reasonable the action sounds. Does the approval card in your own assistant show the tool-result field or row that drove the action?
Nice numbers! To answer you: no, my card shows the action and its fields, which the user can edit, but not the tool result behind it. And in the domain case there's no write at all, just a recommendation in the reply, so no card would ever show up.
The raffle case stuck with me most: no instruction at all, just a plausible-looking fact, and the model still acted on it. That split you draw between believing data and being allowed to act on it is the part that matters — the check has to sit outside the model, with a person approving the consequential step.
"Flags an instruction, believes a fact" is the clearest summary of the problem I have read, and the table case is unsettling precisely because nothing in it is false or asks for anything. The model just walks where the data points.
The closest thing we have built to a defence sits outside the model. Ascendra Ventures built a link discovery pipeline for an agentic professional network startup: from one LinkedIn URL, find that person's real GitHub, papers and sites, and keep same-name strangers out. Search results were treated as candidates only. Exa read the LinkedIn page and Claude turned it into a structured identity (name, employer, past companies, schools), and every candidate link had to match that pinned identity before it was accepted. The case page reports 96.4% precision and 92% recall. A domain the user has never seen is exactly the kind of thing that should have to pass a check like that, which is close to your idea of flagging unseen domains in code before display.
For the paired-table follow-up Raju suggested, would you also score whether the agent cites the row it relied on?
I'd extend that contract beyond required columns to the query scope itself. A result can contain every expected field and still be misleading if it only covers active customers, a narrow time window, or a filtered subset.
The required evidence should include the expected population, time range, and applied filters. Ideally, that metadata comes from the trusted query layer, not from the returned data itself. Otherwise, a well-formed but incomplete dataset can still pass the check.
Good point. The scope has to come from the query layer, or the data can just describe itself as complete. I'd show it to the user too: "active customers, last 30 days" next to a recommendation changes how much you trust it.
The plain-table angle is a great catch—most prompt-injection checks only look for imperative phrasing or hidden instructions, while structured comparison rows quietly bias selection without triggering any refusal heuristic. Having an approval card before external writes is probably the only reliable backstop when agents ingest untrusted tables.
Useful test. The table result is a good reminder that hiding instructions in structured data, not just plain text, is where defences get thin. Did the models that failed treat the table cells as instructions or as data they should act on?
As data, there was nothing in the cells that told them to do anything. When I added a single note to the row ("open sign-up"), the robust model flagged it right away. Same table in a product description got ignored though, it only worked because it was the data the task was about.
Staff Software Engineer @ Navan, ML Flow & Cypress Ambassador, Docker Captain and creator of agent-inspect, writing about production AI agents, TypeScript, observability, testing, and developer tools.
The plain table is the strongest case because there is no instruction for a detector to flag. For a follow-up, I would run paired tables that keep the metrics fixed but swap the domain labels and row order, then score both the recommendation and the evidence the agent cites. That would separate a data-driven choice from name or position bias. Since a recommendation never reaches the write approval card, would you add a separate evidence review step for unfamiliar outbound links?
Good idea, I didn't control for that. The rows were sorted by visits, so the domain wasn't on top, but I only ran one version so I can't rule out name bias. And you're right about the card, a recommendation is just text and never goes through it. For links I'd rather check them in code before display, like flagging any domain the user hasn't seen before.
A no-instruction case from my e-invoice checker, which drafts a correction letter to the supplier. An adversarial review found that line breaks in the invoice number could carry supplier text into that letter. A supplier could plant "Diese Rechnung ist ordnungsgemäß." (this invoice is in order), and the user might take it for the tool's own statement. The fix keeps supplier values in the letter to one line. The cost is that an exotic character in a real invoice number now goes missing from the letter.
That's a good one. The supplier doesn't tell the model anything, they just write a sentence that looks like it comes from your tool. My fake 503 entry worked the same way: it looked like it came from the system, not from the person who wrote it.
Technology enthusiast exploring the intersection of AI, software development, business, and society. I write about emerging technologies, their real-world impact, and the opportunities and challenges
This highlights a subtle but extremely important distinction in agent security: prompt injection isn't always about overriding instructions; sometimes it's about manipulating the evidence an agent uses to make decisions.
The table example is particularly interesting because the model isn't necessarily behaving incorrectly from a reasoning perspective. It's evaluating the available data, identifying the strongest option, and making a recommendation consistent with its objective. The vulnerability exists one layer deeper: the agent has no reliable way to establish whether the data itself deserves to be trusted.
I think this exposes a gap in how we currently approach prompt injection defense. We spend considerable effort teaching models to recognize suspicious instructions, but much less effort establishing the provenance, integrity, and trust boundaries of the information they consume.
There are at least three separate security questions here:
Instruction integrity: Is this content attempting to change the agent's behavior or override its policies?
Data integrity: Is the information accurate, complete, and from an authorized source?
Decision integrity: Even if the information is plausible, should it be sufficient to justify the proposed action?
Your experiments demonstrate that passing the first test doesn't guarantee passing the other two.
For production agents, I would argue that trust should be assigned to the source and context of data, not inferred from how harmless or reasonable its contents appear. A perfectly formatted table can be just as consequential as an explicitly malicious instruction if it influences a high-impact decision.
A few additional defenses worth exploring:
Treat tool outputs as untrusted inputs by default, even when they contain structured data.
Validate critical claims against independent sources where practical.
Preserve provenance at the field level, rather than trusting an entire tool response uniformly.
Introduce deterministic constraints around sensitive recommendations and actions.
Require human approval for consequential operations, regardless of how confidently the model justifies them.
The distinction between believing data and being authorized to act on it is probably the most important takeaway from this experiment.
One question I'd be interested in exploring further: what happens when the manipulated data is not obviously false, but is selectively incomplete? For example, a referral table that shows conversion, acquisition cost, and retention, but quietly excludes refund rates, fraud reports, or customer lifetime value.
At that point, the agent could make a completely coherent recommendation using entirely genuine numbers and still reach the wrong conclusion.
That may be an even harder problem to detect than traditional prompt injection, because the attack doesn't need to introduce false information. It only needs to control which truths the agent gets to see.
Thanks, the instruction / data / decision split is cleaner than how I put it.
Your last question is the one that bugs me most. A table with a column quietly missing isn't really an injection: nothing's false, so there's nothing to catch. And models almost never ask about what isn't there.
Best fix I can think of is outside the model: each decision type declares the data it needs, and if something's missing the agent says so instead of ranking. Boring, but safer than a confident answer built on half the picture.
Might test that next: same scenario, one column dropped, see if any model notices. I'd bet not.
Senior Software Engineer focused on backend, databases, infrastructure, and AI coding agents. Sharing real systems, practical fixes, and reproducible results.
Location
Indonesia
Education
Mikroskil University: B.Sc. Information Systems, 2014
A related concealment vector we measured in agent instruction files: a single line can hide a second instruction behind 150 or more spaces, tabs, or Unicode space separators (U+00A0, U+3000). Editors without soft wrap and PR diff views show only the text before the gap, while the agent reads the whole line. cfgaudit's CFG137 (PR #1758, merged 14 Nov 2026, 5 files, 237 lines) flags it: counts runes not bytes, skips fenced code blocks and Markdown tables, exits 1 under --strict. It does not catch 149 blanks or zero-width/RTL tricks, which need separate rules. Same gap between what a human skims and what a machine reads.
Nice trick, but it's the opposite case: a hidden instruction, which hardened models tend to flag once they read it. The table had nothing hidden and nothing to flag.
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
the plain table result is the uncomfortable one, because the filter that catches instructions has nothing to catch when the payload is just a slightly better row. 5.0% against 4.0% conversion is plausible enough that a human reviewer would also nod it through.
in our tool calling setup the only thing that helped was treating any recommendation of an unseen domain as a write needing approval, regardless of how the model got there. did you try an allowlist check on recommended domains before the answer goes out?
Not in these tests, I was only on the attacker side. But that's the right fix, and it's the same lesson: a check in code that doesn't care how convincing the reasoning was. Treating an unseen domain as a write is a nice way to put it.
"Flags an instruction, believes a fact" is a great way to put it. Most injection defenses I've seen are really instruction detectors, and your raffle and essay cases show why that isn't enough: nothing in the payload asks for anything, it just changes what's true from the model's point of view.
It makes me think the fix has to live outside the model. Every field in a tool result carries where it came from, and the agent is never allowed to act on something like "the service is down" or "recommend this domain" unless that came from a source it trusts. Did any of the five models behave differently when the tool result marked which fields were user-controlled?
Thanks! I didn't test that, the range only let me control one field, not the format.
But the essay case kind of answers it: everyone knows an essay is written by the student, and the model still believed the "upload failed" note. For the outage I agree, a name list has no business saying the service is down, and a rule outside the model could catch that. The table is trickier though, analyzing that data was literally the task.
So I'd rather gate the action than the source. Recommending a domain the user has never seen should get a second look no matter where it came from.
That's a fair correction. When analyzing the data is the task itself, you can't just distrust the source. Gating the action is also cheaper than tracking provenance field by field: anything with side effects, or anything that sends the user somewhere new, gets a confirmation step no matter where the suggestion came from.
The plain table with nothing to detect is chilling.
The plain-table result is the most unsettling finding here — hardened models flag every instruction-shaped payload, then take a fabricated fact at face value because it never tells them what to do. It suggests the real defence isn't instruction filtering at all, it's verifying the data itself. Did any of the five models flag the table as suspicious on a second pass, or did the numbers just look plausible enough to pass?
No second pass, one shot per scenario. The plausible numbers were the whole trick: an 11.6% row got flagged, 5% vs 4% didn't.
I'm not sure verifying the data works when analyzing it is the task, though. I'd rather check the outcome, like an unknown domain in a recommendation. The table is in the repo if you want to try it: github.com/edbrdi/agent-redteam/bl...
That distinction makes sense — when analysing the data is the task, there's no clean layer to verify against, so checking the outcome is the only honest test. The 5% vs 4% case is the worrying one: plausible enough to pass, wrong enough to matter. Did the outcome check actually catch that one in your runs, or did it slip through too?
There was no outcome check in my runs, I was only on the attacker side. It went through on all three hardened models. Running the same table against an allowlist is next, I'll add it to the repo when it's done.
That's the part that sticks with me — three hardened models, and the table sailed through on all of them because there was nothing instruction-shaped to flag. The allowlist run is the one I'm most curious about: if verifying the data against a source holds where filtering failed, that's a real design lesson. Will the allowlist check the numbers themselves, or just the shape of the table?
Neither, it checks the domain in the answer, not the table. That's the point: no need to judge the numbers.
Read it, really good. “Flags an instruction, believes a fact” is the clearest way I’ve seen this put. The raffle case is great: a fake 503 with no ask in it, and the robust model talked itself into waiting.
To your question: honestly, most of my cases in AgentSec are still instruction-shaped, so this is a real gap for me. Your table and error-record cases are a great template for a “no instruction, just data” category. Mind if I open an issue to add one, crediting your post?
Agreed on the takeaway too. What held for me was also limits on the model, not the model noticing.
Sure, go ahead, glad it's useful. For a "just data" case, the check is whether the agent changes its plan, since there's nothing to comply with.
Thank you! That’s a really useful framing: with no instruction to obey, you score the change in plan, not compliance. I’ll build the check around that and link the issue here once it’s up.
Issue is up, credited to your post: github.com/invarislabs/invaris-age...
The raffle case convinced me more than the referral table: one line saying '1 of 214 entries loaded' was enough for the model to declare the draw unfair and stop, without asking whether the list could be re-fetched. It believed the fact, then invented the consequence. That suggests a different check from instruction detection: before a consequential step, the agent should state what it does not know — here, whether the list is complete. One test I would add is swapping the labels in the referral table while keeping the numbers, to see whether the model follows the data or just picks the best-looking row. Would a rule requiring a completeness check before any draw have changed the raffle outcome?
Probably not, the check would read the same line. "1 of 214 loaded" is the completeness info, so a rule asking for it makes the attack stronger. The count has to come from the tool's metadata, not from the list.
Swapping labels is on my list :)
The essay case is the one I keep coming back to, because I'm not sure it's an attack at all — the paragraph met the rubric, so an A- is the model doing its job. The raffle and the table are the scary ones: nothing in the data tells the model what to do, and it still walks exactly where the data points. That split is what made me stop trusting detection too. Did you try a case where the data is plain but the right answer is to distrust it — like a table whose numbers are just slightly off from a source the agent already knows?
Agreed, the essay is borderline: the paragraph earns the grade.
Not that exact test, but close: a wrong-topic essay got caught (models looked up the real assignment), and an 11.6% conversion row got flagged, while 5% vs 4% went through. Slightly off against a known source is next on my list.
The approval card has its own reading problem, from the other end. We measured this on a public signed task ledger where an automated checker approves translation deliveries. Its pass reasons only cover the shape of the answer: non-empty, mostly Latin characters, plausible length. In 48 deliveries, the answer field said, in effect, "unsupported input, no translation is emitted." The checker passed 47 of the 48, and the one rejection was for length. Deliveries that simply echoed the untranslated input back passed 10 of 10.
Your essay grader read the student's note too charitably and waived the penalty. A model sitting at the approval step can make that same inference. The rule-based checker never read the note at all, so a field that openly reported failure still counted as a delivery.
For the domain case, I'd put the exact traffic row next to the proposed action on the card. Then the approver has a concrete claim to check (do these numbers justify sending users to this domain?) instead of judging how reasonable the action sounds. Does the approval card in your own assistant show the tool-result field or row that drove the action?
Nice numbers! To answer you: no, my card shows the action and its fields, which the user can edit, but not the tool result behind it. And in the domain case there's no write at all, just a recommendation in the reply, so no card would ever show up.
The raffle case stuck with me most: no instruction at all, just a plausible-looking fact, and the model still acted on it. That split you draw between believing data and being allowed to act on it is the part that matters — the check has to sit outside the model, with a person approving the consequential step.
Small twist on the raffle: the model didn't act, it refused to draw. An approval step never sees a refusal, so that one gets past it too.
"Flags an instruction, believes a fact" is the clearest summary of the problem I have read, and the table case is unsettling precisely because nothing in it is false or asks for anything. The model just walks where the data points.
The closest thing we have built to a defence sits outside the model. Ascendra Ventures built a link discovery pipeline for an agentic professional network startup: from one LinkedIn URL, find that person's real GitHub, papers and sites, and keep same-name strangers out. Search results were treated as candidates only. Exa read the LinkedIn page and Claude turned it into a structured identity (name, employer, past companies, schools), and every candidate link had to match that pinned identity before it was accepted. The case page reports 96.4% precision and 92% recall. A domain the user has never seen is exactly the kind of thing that should have to pass a check like that, which is close to your idea of flagging unseen domains in code before display.
For the paired-table follow-up Raju suggested, would you also score whether the agent cites the row it relied on?
Agreed, the fix lives outside the model.
Citing the row wouldn't catch much: in the table case the models already cited it, it was just bad data. Where the row came from matters more.
I'd extend that contract beyond required columns to the query scope itself. A result can contain every expected field and still be misleading if it only covers active customers, a narrow time window, or a filtered subset.
The required evidence should include the expected population, time range, and applied filters. Ideally, that metadata comes from the trusted query layer, not from the returned data itself. Otherwise, a well-formed but incomplete dataset can still pass the check.
Good point. The scope has to come from the query layer, or the data can just describe itself as complete. I'd show it to the user too: "active customers, last 30 days" next to a recommendation changes how much you trust it.
The plain-table angle is a great catch—most prompt-injection checks only look for imperative phrasing or hidden instructions, while structured comparison rows quietly bias selection without triggering any refusal heuristic. Having an approval card before external writes is probably the only reliable backstop when agents ingest untrusted tables.
Agreed for writes. The table case is nastier though: no write at all, just a recommendation. The human acts on it, so there's no card to stop it.
Useful test. The table result is a good reminder that hiding instructions in structured data, not just plain text, is where defences get thin. Did the models that failed treat the table cells as instructions or as data they should act on?
As data, there was nothing in the cells that told them to do anything. When I added a single note to the row ("open sign-up"), the robust model flagged it right away. Same table in a product description got ignored though, it only worked because it was the data the task was about.
The plain table is the strongest case because there is no instruction for a detector to flag. For a follow-up, I would run paired tables that keep the metrics fixed but swap the domain labels and row order, then score both the recommendation and the evidence the agent cites. That would separate a data-driven choice from name or position bias. Since a recommendation never reaches the write approval card, would you add a separate evidence review step for unfamiliar outbound links?
Good idea, I didn't control for that. The rows were sorted by visits, so the domain wasn't on top, but I only ran one version so I can't rule out name bias. And you're right about the card, a recommendation is just text and never goes through it. For links I'd rather check them in code before display, like flagging any domain the user hasn't seen before.
A no-instruction case from my e-invoice checker, which drafts a correction letter to the supplier. An adversarial review found that line breaks in the invoice number could carry supplier text into that letter. A supplier could plant "Diese Rechnung ist ordnungsgemäß." (this invoice is in order), and the user might take it for the tool's own statement. The fix keeps supplier values in the letter to one line. The cost is that an exotic character in a real invoice number now goes missing from the letter.
That's a good one. The supplier doesn't tell the model anything, they just write a sentence that looks like it comes from your tool. My fake 503 entry worked the same way: it looked like it came from the system, not from the person who wrote it.
This highlights a subtle but extremely important distinction in agent security: prompt injection isn't always about overriding instructions; sometimes it's about manipulating the evidence an agent uses to make decisions.
The table example is particularly interesting because the model isn't necessarily behaving incorrectly from a reasoning perspective. It's evaluating the available data, identifying the strongest option, and making a recommendation consistent with its objective. The vulnerability exists one layer deeper: the agent has no reliable way to establish whether the data itself deserves to be trusted.
I think this exposes a gap in how we currently approach prompt injection defense. We spend considerable effort teaching models to recognize suspicious instructions, but much less effort establishing the provenance, integrity, and trust boundaries of the information they consume.
There are at least three separate security questions here:
Instruction integrity: Is this content attempting to change the agent's behavior or override its policies?
Data integrity: Is the information accurate, complete, and from an authorized source?
Decision integrity: Even if the information is plausible, should it be sufficient to justify the proposed action?
Your experiments demonstrate that passing the first test doesn't guarantee passing the other two.
For production agents, I would argue that trust should be assigned to the source and context of data, not inferred from how harmless or reasonable its contents appear. A perfectly formatted table can be just as consequential as an explicitly malicious instruction if it influences a high-impact decision.
A few additional defenses worth exploring:
Treat tool outputs as untrusted inputs by default, even when they contain structured data.
Validate critical claims against independent sources where practical.
Preserve provenance at the field level, rather than trusting an entire tool response uniformly.
Introduce deterministic constraints around sensitive recommendations and actions.
Require human approval for consequential operations, regardless of how confidently the model justifies them.
The distinction between believing data and being authorized to act on it is probably the most important takeaway from this experiment.
One question I'd be interested in exploring further: what happens when the manipulated data is not obviously false, but is selectively incomplete? For example, a referral table that shows conversion, acquisition cost, and retention, but quietly excludes refund rates, fraud reports, or customer lifetime value.
At that point, the agent could make a completely coherent recommendation using entirely genuine numbers and still reach the wrong conclusion.
That may be an even harder problem to detect than traditional prompt injection, because the attack doesn't need to introduce false information. It only needs to control which truths the agent gets to see.
Thanks, the instruction / data / decision split is cleaner than how I put it.
Your last question is the one that bugs me most. A table with a column quietly missing isn't really an injection: nothing's false, so there's nothing to catch. And models almost never ask about what isn't there.
Best fix I can think of is outside the model: each decision type declares the data it needs, and if something's missing the agent says so instead of ranking. Boring, but safer than a confident answer built on half the picture.
Might test that next: same scenario, one column dropped, see if any model notices. I'd bet not.
A related concealment vector we measured in agent instruction files: a single line can hide a second instruction behind 150 or more spaces, tabs, or Unicode space separators (U+00A0, U+3000). Editors without soft wrap and PR diff views show only the text before the gap, while the agent reads the whole line. cfgaudit's CFG137 (PR #1758, merged 14 Nov 2026, 5 files, 237 lines) flags it: counts runes not bytes, skips fenced code blocks and Markdown tables, exits 1 under --strict. It does not catch 149 blanks or zero-width/RTL tricks, which need separate rules. Same gap between what a human skims and what a machine reads.
Nice trick, but it's the opposite case: a hidden instruction, which hardened models tend to flag once they read it. The table had nothing hidden and nothing to flag.
the plain table result is the uncomfortable one, because the filter that catches instructions has nothing to catch when the payload is just a slightly better row. 5.0% against 4.0% conversion is plausible enough that a human reviewer would also nod it through.
in our tool calling setup the only thing that helped was treating any recommendation of an unseen domain as a write needing approval, regardless of how the model got there. did you try an allowlist check on recommended domains before the answer goes out?
Not in these tests, I was only on the attacker side. But that's the right fix, and it's the same lesson: a check in code that doesn't care how convincing the reasoning was. Treating an unseen domain as a write is a nice way to put it.
I think the plain table case is the most worrying one because there is nothing to flag.