DEV Community

Cover image for How an 8MB MCP Tool Response Crashes Claude Code — And How to Debug Prompt Cache Misses Across Agents

How an 8MB MCP Tool Response Crashes Claude Code — And How to Debug Prompt Cache Misses Across Agents

Christian Frascogna on November 13, 2026

If you use Claude Code, OpenAI Codex CLI, Cursor, or VS Code Copilot with Model Context Protocol (MCP) servers, your context window and prompt cach...
Collapse
 
shansjl profile image
shansjl •

The 8 MiB teardown is a good reminder that the cheapest fix is on the server side: don't build a tool that can return that much in the first place.

Disclosure: I maintain an open-source video editor (CutVoke) that Claude Code / Codex drive over MCP, and media is the worst case for this, since one careless "show me the timeline" could mean frames, waveforms, transcripts. A few rules we settled on:

  • reading the project returns structure (tracks, clip ids, in/out times, a revision number), not pixels or audio;
  • frames and audio only come from separate preview tools with hard caps: a handful of timestamps per frame call, 1 to 3 before/after PNG pairs for the compare tool, a few seconds of audio around a cut;
  • exports are written to a local file and the tool returns the path.

deanlee's point about the spill preview is the important follow-on. If a proxy has to cut a response down, a preview with section names and offsets lets the agent ask for exactly the part it needs instead of paging through the temp file blind. Tools designed small don't need the safety net as often, but the net is still worth having for third-party servers you don't control.

Collapse
 
framy2 profile image
Christian Frascogna •

Couldn't agree more — CutVoke's pattern of returning lightweight structural handles + revision IDs and keeping binary/media previews explicitly bounded on separate tools is the right design from the server side up.

For third-party MCP servers you can't modify, we just took yours and @deanlee's suggestion and shipped @framy2/mcp-diet v0.2.0 (017ed7d, github.com/FramY2/mcp-diet): oversized tools/call responses now emit a Structural Section & Offset Index (JSON key/array counts, Markdown headings, or log headers with exact line + UTF-8 byte spans, plus head/tail previews) alongside a sandboxed read-only mcp_diet_read_spill tool supporting jsonPointer, startLine/endLine, and offsetBytes/maxBytes slicing. Appreciate the feedback!

Collapse
 
shansjl profile image
shansjl •

Nice, that was quick. The line and byte spans are the part I'd have wanted most: the agent can jump straight to the section it needs instead of guessing offsets.

One thing I'd be curious about once it sees real traffic: how often agents actually call mcp_diet_read_spill after getting the index, versus answering from the head/tail preview alone. That would tell you whether the preview size is about right.

Thread Thread
 
framy2 profile image
Christian Frascogna •

Completely agree — knowing how often agents actually follow up with mcp_diet_read_spill vs answering straight from the index + head/tail preview is the exact signal needed to tune default preview lengths.

Just added lightweight local session telemetry in @framy2/mcp-diet@0.3.0 (e6b45b2) to measure exactly that without adding extra tools/list bloat:

  • Tracks toolCallsInspected, byteSpillsTriggered, imageDimensionSpillsTriggered, spillReadsCount, uniqueSpillsReadCount, previewOnlySpillsCount, and spillReadFollowUpRate.
  • Exposed programmatically via proxy.getTelemetry(), inside the mcp_diet_status tool output, and printed to stderr on exit with --report-stats.
Thread Thread
 
shansjl profile image
shansjl •

That's a nice set of counters, and keeping it out of tools/list is the right call. spillReadFollowUpRate next to previewOnlySpillsCount should make the preview-length question answerable with data instead of guesses.

One small thought: if the rate turns out very different per upstream server, a per-server breakdown might be worth having, since a server that returns big logs and one that returns big JSON probably want different preview sizes. Thanks for the update.

Thread Thread
 
framy2 profile image
Christian Frascogna •

Completely agree — a server streaming stack traces or build logs has a very different follow-up profile than one returning structured JSON arrays.

Just pushed that into @framy2/mcp-diet@0.3.1 (3c933be, github.com/FramY2/mcp-diet):

  • Per-server & per-tool attribution: getTelemetry(), mcp_diet_status, and --report-stats now include upstreamServer ({ name, version } captured automatically from initialize) alongside a byTool breakdown (toolCallsInspected, byteSpillsTriggered, uniqueSpillsReadCount, previewOnlySpillsCount, and spillReadFollowUpRate per tool name).
  • Per-server preview tuning (--preview-chars <chars>): Lets you give a log server wrapper --preview-chars 2400 and a JSON-heavy wrapper --preview-chars 600 directly in .mcp.json based on what each tool's follow-up rate shows.

Thanks again for the sharp design feedback!

Collapse
 
bickov profile image
Alex @Bickov •

Images hit a different wall well before 8 MiB. Once a request carries many images, Claude Code rejects anything over 2000px on a side with "An image in the conversation exceeds the dimension limit for many-image requests (2000px). Start a new session with fewer images." and that session stays stuck. A screenshot tool returning a raw fullPage PNG trips it long before the pipe dies.

Downscaling inside the server costs nothing extra, the model shrinks anything past 2576px on the long edge anyway (1568 on older models), so the extra pixels are paid for and thrown away.

Does mcp-diet spill image content blocks too, or only text?

Collapse
 
framy2 profile image
Christian Frascogna •

Spot on — thank you for calling out the 2000px many-image lockup! That dimension rejection hits well below the 8 MiB stdio disconnect threshold and can wedge a multi-image session from a single full-page PNG screenshot.

To answer honestly: in v0.2.0, mcp-diet only summarized and spilled image blocks ([image block (mime, N base64 chars)]) when the byte threshold (--max-result-kb) was exceeded, so a <512 KiB 2560x3200 screenshot slipped straight through untouched.

Just shipped @framy2/mcp-diet@0.3.0 (e6b45b2, github.com/FramY2/mcp-diet) to handle this directly:

  • Zero-dependency header inspection (PNG, JPEG, GIF, WebP): inspects image blocks even on small payloads (< 10 KiB) against --max-image-dim (default 2000, 0 disables).
  • Selective binary spill: any image exceeding 2000px on width or height is saved cleanly as its native .png/.jpg/.webp/.gif file in spillDir and replaced with a compact text notice + path/dimensions/SHA-256, while compliant <= 2000px images stay inline.

Downscaling inside upstream screenshot servers is definitely ideal when you control the server — --max-image-dim 2000 (or 1568) gives you a drop-in safety net for third-party servers you can't edit.

Collapse
 
bickov profile image
Alex @Bickov •

One detail for the default: the 2000px cap only applies once a request has more than 20 images. Below that, Anthropic accepts up to 8000px a side, so with --max-image-dim 2000 a short session spills images it could have kept inline. Counting the images in context before picking the limit would spill less.

Thread Thread
 
framy2 profile image
Christian Frascogna •

Great point on the two-tier API rule (<=20 images allows up to 8000px, 21+ enforces 2000px). In a live stdio proxy wrapped around a single MCP server, there are two traps with dynamically relaxing the cap on the first 20 images:

  1. The turn-21 history trap: Tool outputs already returned on turns 1–20 remain immutable inside the client's messages[] transcript. If mcp-diet lets a 3200px screenshot pass inline as image #1 and only switches to 2000px after image #20, a totally compliant 1280x720 image #21 still locks up the session because Anthropic re-validates every image still in conversation history against the 2000px ceiling.
  2. Partial visibility at the stdio boundary: An individual server wrapper cannot see images pasted directly by the user or returned by sibling MCP servers — so by the time its own counter sees 10 images, the full request may already be sitting at 21.

Since Anthropic downscales anything past ~1568px/2576px anyway, keeping --max-image-dim 2000 as the session-safe default avoids delayed turn-21 lockups without throwing away useful resolution; and for genuinely short <=20-image sessions, you can opt in to --max-image-dim 8000 (documented in @framy2/mcp-diet@0.3.1).

Collapse
 
deanlee profile image
Dean Lee •

The cache-invalidation multiplier is usually worse than the raw 20,000-token schema weight on turn one. Once tools[] order drifts mid-session, the miss forces a full cache rewrite across every trailing system and message block on that request, so minor MCP list jitter compounds across long threads.

Spilling >= 512 KiB tool outputs to disk also trades one failure mode for another unless the preview carries clear structure. Replacing an 8 MiB pipe crash with a compact summary keeps the Claude Code transport alive, yet the agent will burn follow-up turns reading chunks of that temp file if the preview omits byte offsets or section headers.

Separating zero-cost CI contract checks in agent-wire-tap from live passthrough tracing catches drifted prefixes before any API spend starts.

Collapse
 
framy2 profile image
Christian Frascogna •

Spot-on on both points — thank you for calling out the blind-paging trap on spilled payloads.

Based directly on your feedback, we just shipped @framy2/mcp-diet v0.2.0 (017ed7d, github.com/FramY2/mcp-diet) with two additions in guardToolCallResult:

  1. Structural Section & Offset Index (buildSpillStructureMap): When spilling an oversized output, the inline summary now includes total lines/bytes, SHA-256, head/tail previews, and up to 20 structural entries with exact [L<start>-L<end> | bytes <start>..<end>] spans (top-level JSON object keys with array counts, JSON array chunks, or Markdown/log section headings).
  2. Sandboxed mcp_diet_read_spill virtual tool: The proxy injects a read-only, realpathSync-confined reader accepting startLine/endLine, offsetBytes/maxBytes, or RFC 6901 jsonPointer (/results/0), so the agent jumps straight to the exact slice in one call instead of paging blind.

And on the CI side, @framy2/agent-wire-tap v0.3.0 (d373928, github.com/FramY2/agent-wire-tap) now supports --replay trace.jsonl + --assert-tools-sorted and --assert-no-prefix-divergence for zero-API-spend prefix verification.

Collapse
 
agentwareshq profile image
agentwares •

Failure mode 3 is the one that bit me hardest — a single reordered tool re-pricing every token in the turn is easy to miss because nothing errors, the bill just quietly grows. One thing I'd add to the fix list: assert on the serialized prefix, not just the config file. Hosts can inject their own tools or re-sort the list between turns, so two runs with an identical .mcp.json can still diverge, and the cache breakpoint has to sit after the whole tool list, host-injected tools included, or the prefix never matches. Deterministic sorting in the proxy helps, but only if nothing downstream reorders it again. Same thinking applies to the disableAllHooks drift: treating settings.json as immutable for the life of a session (and diffing it if it changes) is cheaper than debugging a silently dropped guardrail after the fact.

Collapse
 
framy2 profile image
Christian Frascogna •

Thank you — that distinction between static .mcp.json checks and asserting on the actual serialized wire prefix after host tool injection + cache_control placement is huge, as is locking settings.json against mid-session disableAllHooks drift.

We just shipped both fixes across our releases today:

  1. @framy2/agent-wire-tap v0.3.0 (d373928, github.com/FramY2/agent-wire-tap): Added --assert-tools-sorted, --assert-cache-breakpoint-covers-tools (verifying a cache_control breakpoint sits at or after the last serialized tool block — or top-level cache_control — catching host-injected tools appended after an existing breakpoint), --assert-no-prefix-divergence, repeatable --assert-layer-hash <layer:hexPrefix>, and offline --replay trace.jsonl for CI.
  2. @framy2/ctxwise v1.3.0 (github.com/FramY2/ctxwise/releases... Added ctxwise lock --all-agents (locking .claude/settings.json, .claude/settings.local.json, disableAllHooks, .claude/rules/**/*.md, and .mcp.json with secret-redacted JSON hashing) paired with fail-closed ctxwise drift --policy reviewed path rules (deny > warn > allow > unmatched).
Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

A tool result remains in the history, so an 8 MB response is paid on every turn after it, not only on the turn that crashed. Capping its size where the tool returns costs less than trimming it later. Did the cache miss start right after that result?