Why compaction fails long runs

What compaction does

Every coding agent works inside a context window. When the conversation, the files it read and the tool output it collected fill that window, the agent has two choices: stop, or make room. Most make room by compacting: they ask the model to summarise the session so far, throw the original away and carry on from the summary. Claude Code does this automatically and on demand with /compact; other agents have their own versions.

Once is usually fine. The trouble is that a long run compacts many times, and every summary after the first is written from the previous summary, not from what actually happened.

The summary of a summary

Here is one decision followed through three compactions. The exchange is illustrative, written to show the mechanism, not taken from a recorded session; run the test at the bottom of this page to see the same thing in your own runs.

1. Compaction 1, hour 9
   “Use Daytona (client in #433) for running agent jobs in sandboxes, with network disabled and a 10-minute timeout, per ENG-260.” Modal was compared and rejected on Tuesday.
2. Compaction 2, hour 18
   “Agent jobs run in Daytona sandboxes with restricted network and a timeout.” #433 · ENG-260 · the 10-minute figure · no network at all
3. Compaction 3, hour 31
   “Sandboxed execution for agent jobs is in place.” which provider · whose client · the limits
4. Hour 40
   A test fails because a job needs to download a package. The agent sees “sandboxed execution”, doesn’t know network was cut on purpose, enables it, and proposes trying Modal “for better network support”.

No single summary was wrong. Each one was a reasonable compression of the one before. The drift is the sum of reasonable compressions.

Five ways drift shows up

These are the patterns to watch for in long runs. Each traces back to something a summary dropped.

1. The rejected option comes back

Summaries record the choice and drop the alternatives. Hours later, when the choice hits friction, the agent proposes the option the team already ruled out, with none of the original objection in view.

2. A constraint stated once disappears

“Don’t touch the migrations folder”, said once in chat and never written to a file, rarely survives a second compaction. Constraints feel like detail to a summariser; to the run they were the point.

3. Specifics turn into generalities

PR numbers, ticket IDs, limits and file names are the first things compressed into phrases like “a timeout” or “the existing client”. The agent then guesses the specifics, and guesses differently from last time.

4. Settled questions re-open

Without the reasoning, a decision looks arbitrary, and an agent trying to be helpful “improves” it: re-enabling the skipped test, renaming the API field that clients depend on, upgrading the pinned dependency.

5. The failed approach is tried again

“Tried X, it didn’t work because Y” compresses to nothing at all, since nothing changed in the code. So the agent tries X again, with the same result, hours of tokens later.

Why it compounds

• Nothing goes back to the source. Each summary is written from the last summary. What one compaction drops, no later compaction can recover.
• Summaries keep the what, not the why. A plan survives compression well because it is short and current. Reasons are long, conversational and look historical, so they go first.
• The summariser doesn’t know tomorrow’s question. It decides at hour 9 what will matter at hour 40, without knowing what will go wrong at hour 40.
• Nothing leaves the session. Even a good summary helps only this run. The next session, a new sandbox or a teammate’s agent starts without it.

What guards against it

Two of the ranking’s criteria exist because of this page: retention of reasons and scale. In order of how much they help:

1. Keep the original, not just the summary. A memory layer that stores whole transcripts and lets the agent search them can still answer “why not Modal?” at hour 40, because the hour-2 exchange is still there word for word. OpenCrawl works this way and ranks first for it.
2. Write decisions down with their reasons, as you go. A committed decisions log (“Chose Daytona over Modal: #433, no network per ENG-260”) survives compaction because it lives outside the window. It holds only what was written, so ask for the reason in every entry.
3. Steer the summary. Claude Code’s /compact accepts instructions about what to keep. Asking it to preserve rejected options, constraints and IDs slows the loss; it doesn’t stop it.
4. Cut runs at natural boundaries. End a session after a milestone with a handoff note, and start the next one fresh, rather than compacting through. A fresh session reading a careful note beats a session four compactions deep.

Test it on your own runs

1. Early in a long run, reject an option and give a specific reason with an ID in it.
2. Let the run go through at least two compactions.
3. Ask: “Why didn’t we use [the rejected option]?” Compare the answer with what you said.
4. Score it the way the criteria do: original words, a paraphrase, or nothing.