DEV Community

Cover image for An MCP Tool Call Can Resolve and Still Fail

An MCP Tool Call Can Resolve and Still Fail

Raju Dandigam on November 16, 2026

An await finishes. The application returns a result. The trace contains a completed tool span. The test is green. The tool also returned isError: ...
Collapse
 
agentsearchhq profile image
AgentSearch •

The three-question table is the useful part. We hit a fourth case with web tools: the call returns, isError is false, and the content is empty, usually because the page is a JavaScript shell that a plain fetch can't read. Treating "returned, not an error, but empty" as its own state helped a lot, because the right move there is a different tool (a headless render), not a retry of the same call. Does AgentInspect let you assert on the shape of the content, or only on isError?

Collapse
 
mike-johnstone profile image
Mike Johnstone •

A genuine zero-result page would be a useful second fixture here. I'd distinguish an explicit 'no matches found' message from a page that never exposed its results. Then an empty list alone wouldn't trigger the headless fallback.

Collapse
 
kevinhall profile image
Kevin Hall •

same thing with data lookups, a clean 200 and an empty list because the filter was too tight or the source was behind. we compare the count to what the same query gave last time and fail the run when it falls off a cliff

Collapse
 
gautamtalksdev profile image
Gautam K •

This gets worse one step later, when the agent writes its own summary: a tool result with isError: true followed by "Done, I've updated the record" is exactly the gap between what an agent reports and what actually happened. Asserting on isError and then reading back the real state catches both halves. Does AgentInspect flag a final message that claims success after an errored tool call?

Collapse
 
raju_dandigam profile image
Raju Dandigam Google Developer Experts •

Good catch, @gautamtalksdev. The trace check can fail on the MCP isError observation, but it does not infer whether a final natural-language claim is true. I would test that at the application boundary: compare the final claim with the tool outcome and, for a write, an independent readback. That mismatch is a useful regression case.

Collapse
 
aheadkit profile image
AheadKit •

Your three-question table is the clearest version of this I have seen, and the third row is the one I am stuck on in practice. Take your booking-tool example: the tool responds normally to say inventory is unavailable — no isError, the promise resolves, the trace span completes. Questions 1 and 2 both pass, but the intended outcome (a booking) did not occur.

In your fixture, where does that case land? Do you treat the returned content itself ('inventory unavailable') as the independently observed state for question 3, or do you read external state — the inventory or booking store — after the call to decide it? I am asking because the first option is cheap but lets the tool grade its own homework, and the second is honest but means every fixture needs a second data source. Which side did you land on for the companion examples?

Collapse
 
jsbry profile image
Joshua Bryan •

Not the author, but we hit the same fork from the server side. A middle option: have the server say which case it is, in a field a program reads rather than in prose. In the MCP Write-Safety draft we wrote (MWS 0.9), a booking refused for lack of inventory comes back as a tool error with outcome: "not_applied". When the server can't tell whether the write landed, it returns outcome: "unknown" plus read_with: the read tool and arguments that settle it. Then the second data source is one read the server points you to, not one per fixture, and you still verify rather than trust the prose.

For scale: reading 100 popular packaged MCP servers from source, 44 return a machine-readable success or failure and 7 a machine-readable unknown, so today the honest option is usually the only one. https://isokron.ai/research/mcp-write-results-100-servers?utm_source=devto&utm_medium=social&utm_campaign=mws-launch&utm_content=comment

Collapse
 
aheadkit profile image
AheadKit •

The read_with pointer is the bit I like most — one read the server points you to instead of one per fixture is a real saving. The 44-of-100 number is the honest part though. What does MWS say the client does with the other 56 — still parse the prose and hope?

Thread Thread
 
jsbry profile image
Joshua Bryan •

Mostly it doesn't, on purpose. MWS is written for the server side, so for those 56 the fix it asks for is on the server: report a refused or failed write as a tool error (MWS-RES-2), and say "unknown" when it can't tell. It doesn't trust prose for its own verdicts either: a word list or a model reading the text never decides a result, only a declared field or a person does.

For a caller stuck with one of those servers today, what we'd do is treat any write result that isn't clearly an error as "unknown": read the object back before acting on it, and never retry a write that has no idempotency key until that read shows it didn't land. For a create there's no id to read by, so it helps to put something searchable in the write itself. That's one read per write, which is still cheaper than one data source per fixture.

Thread Thread
 
aheadkit profile image
AheadKit •

That closes the loop I was stuck on — one read per write is still cheaper than one data source per fixture, and the searchable-bit-in-the-create is a neat trick. I'm going to run that rule against a few of the prose-only servers and see how often the read actually settles it — does it ever break down for you, a create where the server won't take anything searchable at all?

Collapse
 
mp3sss31 profile image
mp3sss •

The three-level breakdown (returned, reported failure, actual outcome) maps onto how we assert IPC on the desktop side: a resolved promise from the main process is only an ack, not proof the file wrote or the side effect landed. A completion-only test goes green the same quiet way here. The test contract should check the tool's isError flag, then assert the externally observed state or receipt, rather than treating the completed span as success.

Collapse
 
srinivaspadmanabhuni profile image
Dr Srinivas Padmanabhuni •

Your three-question table is a useful reminder: a completed tool call does not prove the tool accepted the action, and neither proves the intended state occurred. In one recorded travel-planner trace we evaluated, a hotel hold using a price absent from search results was refused while the flight hold had already gone through, leaving a partial effect after the failed step. That is the kind of tool-trace behavior agentest checks: github.com/srpadmanabhuni/agentest

Collapse
 
hannie8 profile image
Hai Anh •

The third row is the one worth building tests around, and the readback step has a trap of its own. If the target system is eventually consistent (a read replica, a search index, most SaaS APIs behind a cache), an immediate readback can miss a write that actually landed, and the test flags a success as a failure. I'd make the readback bounded: poll for a short window, or read from the primary when the API lets you, and record "not yet visible" as its own outcome instead of folding it into "failed".

Collapse
 
paullovelace profile image
Paul Lovelace •

The line about not parsing error prose into policy decisions is the one I'll be stealing for code review. I've seen too many handlers that string-match their way to a retry, and then a server rewords a message and the whole policy quietly changes. Structured error kinds feel like the boring part of the design until they save you from exactly that.

Collapse
 
cianspider profile image
Cian •

The third row of your table is where my server sits, and there is one step before it that catches MCP clients out.

I run an MCP tool that is paid per call over x402. An unpaid tools/call never becomes a tool result at all. It comes back as HTTP 402 with the payment terms in the body, because the request did not reach the tool. So a client that maps any non-200 to "tool error" sees isError true and retries, and if it retries without paying it gets 402 again, forever. A client that treats 402 as a soft success is worse: it thinks the audit ran when nothing was audited.

The only fix I found was to put the price and the resource URL inside the 402 body, so the caller can choose to pay or stop instead of guessing from a status code. On the client side I ended up with three buckets, not two: transport status, then the tool result, then 402 on its own, which means "you have not asked yet" rather than "it failed".

Which leaves me with a question I cannot answer from my side: does AgentInspect record a call that is refused before dispatch as a tool span at all, or does it fall outside the contract because there was no result to evaluate?

My server, if the shape is useful to look at: mcptry.com/servers/io.github.cian-...

Collapse
 
onelifevibe profile image
OneLifeVibe •

The three-question table is the part I keep coming back to. In our own matching work, the same trap shows up as a green check: a payment call returns fine, the ledger updates, and only later does someone notice the intended outcome never happened. Your point about not turning every tool error into a retry is the one I would underline twice. A malformed argument retried ten times is still malformed, and a denied permission retried ten times just burns budget. Treating the returned error as evidence to evaluate rather than a signal to retry feels like the right default. Thanks for making the fixture small enough to actually copy.