This is an English version of an article first published in Japanese on Zenn.
Introduction
When you do long work in Claude Code, compaction kicks in once the conversation gets long, and the conversation so far is replaced by a summary. After that, Claude no longer remembers something we decided a little earlier. I explain it again. The next compaction wipes it out again.
I expect a future where people no longer do the design work by hand, and an LLM builds straight from rough requirements, fixing things as it goes. To experience for myself what problems come up there, I'm building a VS Code extension for Claude Code in exactly that way.
The problem I hit again and again was this one: decisions disappearing. The only place my decisions live is the conversation, so when compaction loses them, there's nowhere else they survive. Conversations also tend to get long, so compaction happens many times. So I decided to add a feature myself that fills in what compaction loses. What I had in mind was carrying my messages forward verbatim, separately from the summary.
This isn't only about writing code. When you write documents or prepare materials, most of what you decide ends up in the finished product, but decisions like "don't use this word" or "leave this topic out" don't. Things you decided to leave out aren't written anywhere. Only the conversation holds them.
How do you make up for it so nothing is lost? To decide that, I ran an experiment using real development conversations. I found 4 things.
- Even a single compaction loses information. But the instructions I gave were not lost at all after the first one.
- Instructions disappear when compaction repeats. After 3 rounds, I had lost 33% of the instructions I tracked in one conversation and 57% in the other.
- The cause was that my messages from earlier rounds were not carried into the next summary.
- When I carried all my messages forward verbatim, 0 instructions were lost after 3 rounds. You can reproduce the same thing with a single prompt attached to /compact, as long as the conversation doesn't have too many messages.
First, here is a prompt you can paste as is. Paste it together with the leading /compact and run it. This is an English translation of the Japanese prompt whose effect I measured; I did not measure the translation itself. Character counts in this article are counts of Japanese characters. The 1000 in the prompt was set for Japanese text.
/compact At the end of the summary, add a section titled "User messages (verbatim)". In it, copy every message the user sent in this conversation, in order from first to last, without omitting any and without rephrasing, verbatim. If the previous summary has the same section, copy all of its contents verbatim at the start. Only for a message longer than 1000 characters, you may omit the middle, keeping the beginning and the end. Write all other sections as usual.
In conversations of up to about 90 messages (the two I measured had 72 and 86), not a single message went missing after 3 rounds. How to attach the same prompt to automatic compaction is covered later, in "Automatic compaction".
How I ran the experiment
I used 2 real conversation logs that had grown long while I built the extension (conversation A and conversation B, both in Japanese).
- Cut the conversation log at some point.
- Run /compact there (round 1).
- After the summary, append the actual continuation of the conversation as it was.
- Run /compact again (round 2). Repeat once more (round 3).
| Conversation | Round 1 compaction | Round 2 compaction | Round 3 compaction | My messages (cumulative) |
|---|---|---|---|---|
| A | 227k → 11k | 43k → 6k | 34k → 7k | 24 → 42 → 72 |
| B | 388k → 8k | 63k → 7k | 16k → 5k | 36 → 83 → 86 |
The numbers are tokens, measured with plain /compact. Round 1 shrinks a 200k–400k-token conversation to around 10k. What round 2 compacts is "the round 1 summary" plus "the conversation after it."
What I counted was whether the instructions and decisions I had given up to that point could still be read after compaction. I picked 18 items per conversation, and after each compaction, had a separately run Claude score them. I repeated the same procedure 3 times each.
The scope I checked: Japanese conversations from one user (me), with Opus 5.5 as the only model doing the summarizing. What I looked at was whether something was written down after compaction. I did not check whether Claude then acted accordingly.
Conditions Scoring categories Separately, I also counted decisions added that I had never made. Scorers Scoring was left to a separately run Claude (Opus 5.5), with the method names hidden. The 2 methods that appear in the "Fix" section (programmatic carry-over and prompt-based copying) were built later, so they were scored in new scoring runs, separate from the original ones, with a different scoring Claude for each. To see whether the grading standard had drifted from the earlier scoring, I mixed in 6 already-scored summaries for each, in a form that didn't reveal which they were. Full agreement with the earlier scores was 89% and 95% respectively, and agreement on "lost / not lost" was 95% and 99%.Experimental conditions and scoring
CLAUDE.md. Whatever was already in the conversation log stayed as it was.
What compaction loses
How much one compaction loses overall
First, the overall picture. With the plain summary, even a single compaction loses information. I counted what survived the first compaction, including things other than my instructions.
| What I counted | After round 1 |
|---|---|
| Facts needed to continue the work (what's done, remaining work, the content of proposals I accepted, etc.) | Clearly written 70%, partly 23%, missing 7% |
| File names and IDs that appeared 3 or more times in the conversation (conversation A) | 46–71% kept |
| Same (conversation B) | 21–23% kept |
30% of the facts were incomplete or gone. Of the file names and IDs, 30–50% were lost in conversation A and about 80% in conversation B. The feeling that "it forgets things after even one compaction" matches the numbers.
How this table was counted
But my instructions were not lost in round 1
Among all that, the instructions I gave were the one thing not lost at all in round 1. They start disappearing from round 2.
| Share of instructions lost | After round 1 | After round 2 | After round 3 |
|---|---|---|---|
| Conversation A | 0% | 24% | 33% |
| Conversation B | 0% | 18% | 57% |
Here are examples of messages lost by round 3.
Bug fixes first
Just make sure the design doesn't get messed up
Both are short remarks, but in the flow of the conversation with Claude, they were clear instructions. When I repeated the 3-round procedure 3 times, the first was lost all 3 times, and the second was lost twice and kept once. Whether something is lost changes from one compaction to the next.
That nothing was lost in round 1 did not match my experience. I had felt that even one compaction made Claude forget what we'd decided. The numbers suggest 2 reasons.
- Much of what I felt was forgotten wasn't instructions. Which files were changed, what's done and what's left. Facts and file names like these are lost from round 1, as the table above shows. Even if the instructions survive, you end up explaining the background again.
- In long work, compaction doesn't happen just once. It happens many times without you noticing. Instructions start disappearing from the second one.
The cause: earlier messages were not carried forward
Before the experiment, here's what I thought. My messages are short and lean on earlier context. In conversation A, all 72 messages together come to 3,403 characters, under 50 characters per message. Many are one-liners like "in Japanese" or "A is fine." From the summarizer's side, it's hard to tell which ones are important decisions. That's why they get dropped, I assumed.
When I counted what was actually in the plain summary, that turned out to be wrong.
Claude Code's summary already has a section called "All user messages." Compaction does try to list my messages. So I counted how many of my messages appeared in the summary with their exact wording.
| Round | Conversation A: messages new in that round | Conversation A: earlier messages | Conversation B: messages new in that round | Conversation B: earlier messages |
|---|---|---|---|---|
| Round 1 | 18 of 23 | — | 30 of 34 | — |
| Round 2 | 15 of 18 | 0 of 23 | 45 of 47 | 0 of 34 |
| Round 3 | 29 of 30 | 0 of 41 | 2 of 2 | 0 of 81 |
(Messages shorter than 6 characters are excluded, since they match by chance as part of other text. That's why the counts are slightly lower than in the table in "How I ran the experiment" (my messages, cumulative). This is one of the 3 runs; the other 2 were almost the same. The right side of the figure is the fix described in the next section.)
Of the messages that were new in that round, short remarks included, most appear verbatim. The summary was not dropping short messages because it couldn't tell what they meant. That's why instructions aren't lost in round 1.
Earlier messages, on the other hand, did not survive verbatim, not even one.
In other words, the summary keeps that round's messages. But it does not copy the list from the previous summary into the next one. The round 2 compaction reads the round 1 summary, but the list of messages there gets rewritten into the body as "earlier discussion." Any decision not picked up at that point no longer exists anywhere.
You could argue they were lost simply because the conversation got longer. So I took the same conversation up to the round 3 point, compacted it only once, and put the results side by side.
| Instructions lost (out of 54 items, cumulative over 3 runs: 18 items × 3) | Compacted once | 3 rounds |
|---|---|---|
| Conversation A | 0 | 18 |
| Conversation B | 2 | 31 |
At the same length, large losses happened only when compaction was repeated. Even a single compaction loses a little once a conversation gets this long (the 2 in conversation B).
Fix: carry all messages forward, verbatim
If the cause is "not carrying forward," the fix is "carry forward." My guess about the cause was wrong, but the fix I had in mind (carry all messages into the next round) turned out to address the real cause directly.
I tried 2 ways of carrying messages forward.
- Programmatic carry-over: a program extracts my messages from the conversation log, attaches them to the summary and passes them on. This is what I put in the extension.
- Prompt-based copying: just add a prompt after /compact asking it to copy all messages verbatim, including the ones in the previous summary. No program involved.
| Share of instructions lost | After round 1 | After round 2 | After round 3 |
|---|---|---|---|
| Plain summary (conversation A) | 0% | 24% | 33% |
| Plain summary (conversation B) | 0% | 18% | 57% |
| Programmatic carry-over (conversations A and B) | 0% | 0% | 0% |
| Prompt-based copying (conversations A and B) | 0% | 0% | 0% |
With both methods, 0 instructions were lost after 3 rounds. Since the messages are there verbatim, that's to be expected.
Totals over 3 runs. The item counts include only instructions I had given up to that round. Whatever is missing from "kept correctly" plus "lost" to make 100% is "quote only," "stale" and "scope widened." "Decision list" is the design where the AI writes a list of decisions (explained under "Another design I tried" in this section). "Comparison variant" is a form made only for comparison: the plain summary with the raw messages attached. Unlike programmatic carry-over, it has neither the one sentence asking the summary to keep its sections nor the step that trims long messages. Conversation A Conversation B "Kept correctly" moves by a few points when the scorer changes. The differences among the 3 forms that carry messages forward can't be firmly attributed to the methods. For "lost," scorer agreement was 95–99%, so that number is stable.Detailed results (kept correctly % / lost %)
Round
Items (total over 3 runs)
Plain summary
Decision list
Comparison variant
Programmatic carry-over
Prompt-based copying
1
24
100 / 0
100 / 0
100 / 0
100 / 0
100 / 0
2
45
71 / 24
80 / 13
93 / 0
98 / 0
98 / 0
3
54
67 / 33
80 / 20
81 / 0
94 / 0
94 / 0
Round
Items (total over 3 runs)
Plain summary
Decision list
Comparison variant
Programmatic carry-over
Prompt-based copying
1
18
89 / 0
89 / 0
89 / 0
94 / 0
94 / 0
2
51
80 / 18
71 / 10
82 / 0
82 / 0
96 / 0
3
54
33 / 57
65 / 26
85 / 0
74 / 0
81 / 0
What I built into the extension
There's one idea behind it: reduce the parts that depend on how the AI writes.
- Leave the summary itself to /compact. The only thing I add is one sentence asking it to keep the summary's usual sections and to put each heading on its own line. I don't ask it what to keep.
- Extract my messages from the conversation log mechanically. Instead of having the summarizing AI copy them, a program reads the log and pulls out only the user's messages, with timestamps. Since they never pass through the AI, nothing is rephrased or dropped.
- Carry forward all messages from earlier rounds too. This is the part the plain summary wasn't doing.
- Trim the middle of long messages only. For a message over 1000 characters (pasted logs or code, for example), the beginning and the end are kept. Where something was cut, it notes how many characters were cut and that the full text is in the original log.
- After the summary, pass along the list of messages, together with notes on how to read it.
The notes say these 3 things:
- The user's messages are the original text in time order, not rephrased. For wording and intent, this original text is authoritative.
- However, the current state of each decision (in effect, withdrawn, not yet done) is judged from the summary. The original text also includes messages that were later withdrawn.
- If the original text and the summary disagree, check with the user.
The summary and the original text get separate roles. What was said comes from the original text; where things stand now comes from the summary.
Another design I tried: having the AI write a decision list
Instead of carrying the original messages, I also tried a design where the AI writes out a list of decisions at compaction time, noting "who decided" for each one. More survived than with the plain summary (instructions lost after 3 rounds: 20% in conversation A, 26% in conversation B), but decisions I never made sometimes went into the list as "decided by the user" (over rounds 1 to 3 combined, 5 in conversation A and 6 in conversation B; 0 with the plain summary).
A decision list written by the AI is a second record of the decisions, written by the AI. Unless you can compare it with the original messages, you can't tell whether it's right. I wrote in an earlier article (in Japanese) that adding copies leaves them behind as stale versions that get in the way of later work. So I stopped having it write a list and switched to carrying the original messages as they are.
Trying it with a single prompt
You can try the same thing without installing the extension. How you attach the prompt differs between running compaction yourself and letting it run automatically.
Manual compaction
Paste the prompt from the beginning of this article, together with its leading /compact, and run it. Do this every time. If an automatic compaction comes in between, that round behaves the same as the plain summary. For long work, also set up the automatic case below.
The Japanese original of that wording is what I used in the experiment. The key is the third sentence: "If the previous summary has the same section, copy all of its contents verbatim at the start." It asks the summary to carry forward what the plain summary did not.
In conversations A and B, in all 18 summaries (2 conversations × rounds 1–3 × 3 runs), every one of my messages appeared verbatim, apart from whitespace differences (72 and 86 messages at round 3).
Automatic compaction
With automatic compaction, you can't add text after a command the way you can with /compact. There are 2 ways.
Write it in CLAUDE.md. Create a section called "Compact Instructions" in the CLAUDE.md at your project root and put the prompt there.
# Compact Instructions
At the end of the summary, add a section titled "User messages (verbatim)". In it, copy every message the user sent in this conversation, in order from first to last, without omitting any and without rephrasing, verbatim. If the previous summary has the same section, copy all of its contents verbatim at the start. Only for a message longer than 1000 characters, you may omit the middle, keeping the beginning and the end. Write all other sections as usual.
With this section in place from the start of the conversation, I set things up so that automatic compaction would kick in at the round 1 point of conversation A, and all 24 messages appeared verbatim. Same result as the manual case.
However, when I added it partway through the conversation, it didn't make it in time for the next compaction. The messages section wasn't created, and only 19 of the 24 messages appeared verbatim. CLAUDE.md is loaded when the conversation starts, and the next time it's reloaded is after compaction. If you use CLAUDE.md, write it before you start the conversation.
If you want it to reach the summary reliably, pass it through a hook. Claude Code has a hook that runs right before compaction (PreCompact), and the text this hook outputs is added to the prompt for the summary, on top of Claude Code's built-in summarization prompt (official docs). Put this in your project's .claude/settings.json. If the file already exists, add the PreCompact entry inside hooks (if PreCompact is already there, add one more item to its list; if there is no hooks yet, add hooks along with it).
{
"hooks": {
"PreCompact": [
{
"matcher": "auto",
"hooks": [
{ "type": "command", "command": "cat \"$CLAUDE_PROJECT_DIR/.claude/compact-prompt.txt\"" }
]
}
]
}
}
Save the prompt in .claude/compact-prompt.txt:
At the end of the summary, add a section titled "User messages (verbatim)". In it, copy every message the user sent in this conversation, in order from first to last, without omitting any and without rephrasing, verbatim. If the previous summary has the same section, copy all of its contents verbatim at the start. Only for a message longer than 1000 characters, you may omit the middle, keeping the beginning and the end. Write all other sections as usual.
With this setup too, I set things up so that automatic compaction would kick in at the same point, and all 24 messages appeared verbatim. Every time automatic compaction runs, the hook's output is added to the prompt for the summary. For a manual /compact, you still attach the prompt by hand as before. After setting it up, start a new conversation before relying on it. The hook leaves no trace of having run, so after the first automatic compaction, check that the summary has the "User messages (verbatim)" section. The command uses cat, so on Windows you need Git Bash.
In a conversation with many messages, it broke down
For the manual case, I checked whether it holds with many more messages, using another real conversation (conversation C). It has 133 messages, about 12,000 characters in total. It includes long messages, such as one where I pasted research results from another AI. As before, I compacted 3 rounds and ran each of the 2 methods once.
Prompt-based copying had every message after round 1, when there were 45 messages. It broke down at round 2, when there were 114. After round 3, it looked like this:
| What I looked at | Prompt-based copying | Programmatic carry-over |
|---|---|---|
| Messages of the 133 that appeared verbatim | 129 | 133 |
| Messages dropped entirely | 1 (34 characters) | 0 |
| Messages of 1000 characters or less that had their middle cut | 2 (888 and 610 characters) | 0 |
| The 2,616-character message | Middle cut, replaced with a summary written by the AI | Beginning and end kept, with the number of characters cut noted |
4 of the 133 broke, which isn't many. But the way they broke is bad.
- A dropped message doesn't come back. The one dropped in round 2 was still missing in round 3. The next round only copies the previous list.
- The AI's paraphrase goes into the cut. A section meant to carry the original text verbatim got a summary written by the AI mixed in.
- The prompt wasn't followed. It says only messages longer than 1000 characters may be cut, yet messages of 888 and 610 characters were cut too.
As long as the AI is doing the copying, nothing guarantees that messages won't be dropped as they pile up.
How the 2 methods differ
| Prompt-based copying | Programmatic carry-over | |
|---|---|---|
| What you need | One prompt | A program (the extension) |
| Conversations with 70–90 messages | None dropped | None dropped |
| The conversation that went past 110 messages (C) | Drops and cuts appeared | None dropped |
| Size after the round 3 compaction (conversation A) | About 6,600 tokens | About 13,000 tokens |
| Same (conversation B) | About 5,100 tokens | About 10,700 tokens |
Sizes are medians of 3 runs. For programmatic carry-over, the attached part was measured separately and added to the summary size.
The plain summary is about 7,000 tokens for conversation A and about 4,600 for conversation B. Prompt-based copying came out at almost the same size as the plain summary, even though it adds the messages. My reading is that the other sections of the summary got shorter to make room for the message list. In round 3 of conversation A, the "Primary Request and Intent" section of Claude Code's own summary went from about 2,000 characters to about 400.
I also checked whether that shortening cost information other than instructions. Separately from instructions, I picked 15 facts per conversation needed to continue the work and counted them after the round 3 compaction. The share "clearly written" was 49% for the plain summary versus 43% for prompt-based copying. The gap is small, within the noise. But for one category, "the content of proposals Claude made and I accepted," the plain summary had 50% and prompt-based copying only 33%. Programmatic carry-over had 50% in that category too, no lower than the plain summary, and 56% overall.
From the original conversations, I picked 15 facts per conversation that aren't my instructions but are needed to continue the work (what's done, remaining work and its conditions, the content of proposals Claude made and I accepted, the current task and the next step). The Claude that picked them was not shown the summaries. These were scored on the round 3 summaries. Two scoring Claudes scored all 18 summaries (270 items in total, 91% full agreement).What got lost when the summary got shorter
Method
Clearly written
Partly
Missing
Plain summary
49%
23%
28%
Programmatic carry-over
56%
37%
7%
Prompt-based copying
43%
48%
8%
I wanted the option that doesn't depend on how well the AI copies, so I put programmatic carry-over in the extension. It's about twice the size, but it doesn't break down as the conversation grows. If you want to try it first on a conversation with fewer messages, one prompt is enough.
Weaknesses that remain even when carrying the original text
Carrying the original text did not make everything work.
The quote is there, but you can't tell whether it still holds. In round 3 of conversation B, this was 20% with programmatic carry-over and 19% with prompt-based copying. For example, the message "no need to make it fancy" is there. But what not to make fancy can't be known unless the summary says so. The message isn't lost. It just isn't usable on its own. The weakness of short, context-dependent messages showed up here, not in messages being dropped.
A withdrawal sometimes isn't reflected. In conversation B, there was a case where I decided something and withdrew it a little later. The original text includes both the decision and the withdrawal. Even so, with programmatic carry-over, all 3 runs wrote the decision as still in effect. Even with the decision and its withdrawal sitting apart in the list, the single sentence written in the summary is read as stronger. The same case also came out stale at times with the plain summary. In the 3 runs of prompt-based copying, that didn't happen (2 runs wrote the withdrawal, and in 1 the state couldn't be read). It's a weakness that appears or not depending on how the summary is written.
Both happen where the original text and the summary meet. Not dropping messages and knowing each message's current state turned out to be separate problems. The note "if they disagree, check with the user" is the safety net for that.
How this relates to prior research
Alongside the experiment, I looked for papers. Instructions getting lost in compaction has already been measured.
- Lost in Context Compaction (arXiv:2608.11242): of constraints inserted into a conversation, on average 17% remain after compaction. If the constraints are extracted separately and added after the summary, over 90% remain.
- Governance Decay (arXiv:2606.22528): violations rise from 0% to 30% after 1 compaction, and to 78% after 4. Keeping the original text fixed brings it to 0%.
- Benchmarks Are Surprisingly Compaction-Friendly (arXiv:2610.31778): of 269 constraints collected from 44 real Claude Code sessions, 19% are lost in the next compaction.
- Compact-Bench (arXiv:2610.31710): LLM summaries keep only 16% of recurring paths and IDs, even in the best case. Mechanical extraction keeps 79%.
The conclusion that keeping the user's messages in their original form prevents loss points the same way as these.
Where my results look different is round 1 instructions. The papers report that one compaction loses 20% to 80% of constraints, or raises violations to 30%. Here it was 0. The conditions differ, though.
- 2 of the 4 summarize synthetic conversations with summarization setups the authors built. Not Claude Code's /compact.
- Claude Code's summarization prompt has a section that lists the user's messages (it's shown in Compact-Bench). My reading is that instructions weren't lost in round 1 here because the messages appeared verbatim in that section. Of the 2 papers using synthetic conversations, the 1 whose summarization prompt I could check had no such section.
- The closest is the paper that looked at 44 real Claude Code sessions. But it doesn't say whether those compactions were the first in the conversation or later ones. If later ones were mixed in, the 19% figure doesn't contradict my results.
None of these 4 followed real development conversations through 1, 2 and 3 rounds under the same conditions.
Side by side, the closest comparisons look like this: For the round 1 gap, possible factors are differences in the summarization prompt, the summarizing model, the conversations, and what was counted (whether it's written down vs whether it's followed). I haven't isolated which one matters. I could not check the summarization prompt Governance Decay used.Conditions of each paper
Paper
Conversations summarized
Summarization tool and model
Rounds
What was counted
Lost in Context Compaction
Mostly synthetic (about 100k tokens)
The authors' setup. Non-Claude models
1
Whether the constraint survives in meaning in the summary
Governance Decay
9 synthetic scenarios
The authors' setup. Summaries with tight length limits. 7 models
1 and 4
Whether the AI takes a forbidden action after compaction
Benchmarks Are … (analysis of 44 sessions)
44 real Claude Code sessions
Claude Code. Version and model not stated
1 (the next compaction). Which round in the conversation is not stated
Whether the constraint survives in the next summary
Compact-Bench
146 coding sessions (143 real, 3 synthetic)
The summarization prompts of 5 products, run on the authors' setup
1
Whether the string appears verbatim in the summary
This article
2 real conversations (200k–400k tokens, Japanese)
Claude Code's own /compact. Opus 5.5
1–3
Whether my instructions can be read after compaction
What's measured
Papers' numbers
My numbers
User instructions and constraints (1 compaction)
17% remain / 19% lost / violations rise to 30%
0% of instructions lost
User instructions and constraints (after repeats)
Violations at 78% after 4
After 3, 33% and 57% of instructions lost
File names and IDs (1 compaction)
16% or less remain
46–71% and 21–23% remain
Here's what I didn't find in what I read:
- Measurements that track how much of the user's messages carries over with each repeated compaction.
- That Claude Code's summary includes that round's messages but doesn't carry forward earlier rounds'.
- That a decision list written out by the AI keeps the AI's proposals as the user's decisions.
- How decisions the user withdrew are handled.
- Japanese conversations.
I don't know why the summary doesn't carry forward the list of earlier messages. The list grows with every compaction, so maybe it's to keep the size down. If you paste logs or code into the conversation, carrying everything forward keeps all that pasted text, which defeats much of the point of summarizing. Part of why carrying everything works for me is that my messages are short. If anyone knows the real reason, I'd love to hear it.
Closing
When compaction repeats, decisions disappear: after 3 rounds, 30% to 60% of my instructions were lost. Not because short messages get dropped, but because the summary wasn't carrying earlier rounds' messages into the next one. I carried all my messages into the next round without rewriting them, and left the summary to /compact as before. That alone brought instructions lost after 3 rounds down to 0.
I wrote this article while talking with Claude, too. Decisions like "don't use this word" or "leave this topic out" leave no trace in the text. A compaction kicked in while I was writing, but because my messages were carried over verbatim, those decisions held.
A word about the tool. Both the conversations used in this experiment and the feature that fills in what compaction loses come from LAISORA, my VS Code extension for Claude Code (VS Code Marketplace). When something bothers me while using it, I turn that straight into a feature and check on my own conversations whether it really got better. Even without specialist knowledge, you can build a tool just for yourself to answer your own questions. I think that's a great thing about this moment.
If you've felt after compaction that "something we decided is gone" or "I'm being told I decided something I never asked for," check whether your own messages are reaching the next round in their original form. If they are, you can at least compare the summary against what you actually said. Paste and run the prompt from the beginning of this article, and you can try it right away.


Top comments (12)
Useful measurement, thank you. The part that matches my experience is that a decision to leave something out lives only in the conversation. I stopped trusting the summary for those and moved them out of the chat. Each decision gets one dated line in a short list (mine is headed “do not re-argue”), and every new session reads that list before it does anything. A compaction or a fresh session can then lose the wording of the chat without losing what was settled. I built that into Atelier, a Mac app I make for running Claude Code sessions, but a plain file the session must read first does the same job. Do you plan to measure whether Claude acts on the carried messages after round 3, not only whether they are still written down?
Thank you, and you read it correctly: I only counted whether the messages were still written down. Whether Claude acts on them is not measured, and I don't plan to measure it. Whatever the result, it would describe today's models, and I expect that part to change as they improve, most likely for the better.
In daily use I'm also not running into it. This is a feeling, not a measurement, but I've stacked more than 10 handoffs in one line of work without hitting a problem. That is with the programmatic carry-over in my own extension, not with the pasted prompt.
On the list: I moved away from notes because, in my experience, the model forgets to write them down, and the ones it does write go stale. With documents especially, I often say "actually, this way is better" a little later, and then the note needs versions of its own. Carrying the original text doesn't make that go away either. When I have a model re-read meeting minutes where a conclusion was later reversed, it only works if I shape the search prompt carefully, and I'm still working that out. How do you handle a reversed decision in your "do not re-argue" list? No rush.
Sorry for the slow answer, I missed the question at the end. I don’t keep both versions. Each decision is one line with its date, and when I change my mind the old line is rewritten, not followed by a new one, so a session never has to work out which of two lines wins. When the reversal is the point, the line says so: “Rejected: X, do Y instead”, which also stops the old idea coming back. The list stays short because it only holds what is true today; the history lives in the session notes, which nothing reads first. Your stale-notes point is real though: a line only stays right if rewriting it is part of making the change.
Thank you, that makes sense. Rewriting the line in place, and writing "Rejected: X, do Y instead" when the reversal matters, keeps the list focused on what is true today, without two conflicting entries competing.
In my experience, leaving the updates to the agent leads to missed changes, while updating by hand breaks momentum. So I'm curious about that exact challenge you mentioned: when you pivot, do you rely purely on updating the line by hand, or have you built a lightweight trigger into your workflow or Atelier to catch it? No rush.
Neither, mostly. Updating by hand broke my momentum too, so the session writes it: before a session ends, its handover step asks for the decisions settled in that conversation, and the agent records each one as a dated line, naming what it reverses, like “14 Nov: uses an API key after all, reverses 27 Oct”. I only read the list when the next session starts. What it still misses is a pivot I make in my head and never say in the chat.
Thank you, this is a lovely idea.
Our setups differ in where the notes live. Mine starts from a TODO list: each work item has its own notes field, and that is where I keep things in enough detail to work from. Within a single line of work, nothing gets lost because the handoff carries the conversation itself along with the compact summary. Where it falls short is when a separate session opens that TODO item and reads only the notes. It doesn't always pick up the current state accurately. That's where your dated lines that name what they reverse will help, and I'd like to adopt that approach there.
Thanks again for walking me through it.
Losing earlier instructions after a few compactions is why I keep important constraints out of the chat. With the approved plan as a file in the repo and past decisions in a memory server, a fresh or compacted session reads them again instead of relying on the summary. I built QualityLayer for this: github.com/maxritter/qualitylayer
Thanks for sharing QualityLayer. Others in this thread have suggested the same, and I agree: keeping the plan and past decisions outside the chat is the starting point. What I'm after is the next step, running it without adding maintenance overhead.
Someone has to write those entries and keep them current. When several people share that memory store, one wrong entry reaches every session. But if people have to approve every entry to prevent that, human review becomes a bottleneck as agents get faster. In QualityLayer, who writes and updates the decisions in the memory server, and how do you keep them accurate without a person checking each one? No rush.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.