Last month I watched a single fact make a five-hop journey across an org. An engineer wrote a careful incident review: what broke, why, and the one caveat that mattered (the fix was a stopgap and would not survive the next traffic spike). A model summarized the review for the team channel. The channel summary got rolled into the weekly ops report. The ops report got summarized into an exec brief. And the exec, short on time, pasted the brief into an assistant and asked for the TL;DR. By the time the information reached the decision, the caveat was gone. The stopgap shipped as the fix.

Nobody in that chain did anything unreasonable. Every hop was a sensible person making a long thing shorter for a busy person. That is what makes the pattern dangerous: it is summaries all the way down, and each individual summary looks like good citizenship.

Compression compounds

Every summary is lossy compression. That is the point of a summary; it drops what looks unimportant. The trouble is that caveats, edge cases, and dissent are statistically rare in a document, structurally subordinate, and therefore look exactly like the unimportant stuff. They get dropped first. The research backs the intuition: studies of LLM summarization keep finding that omission outpaces hallucination; the model is far more likely to quietly leave the load-bearing sentence out than to make one up. And work on iterative generation (the academic version of the telephone game) shows the distortion compounds with every hop. The tails of the distribution disappear, and the caveat lives in the tail.

The caveat is always the first casualty. It is rare, it is subordinate, and it is the single most expensive thing in the document to lose.

Matthew Purdon

There is a Tom Fishburne cartoon from 2023 that nailed the loop before most of us were living it: one worker uses AI to inflate a bullet point into a long email he can pretend he wrote; the recipient uses AI to deflate the email back into a bullet point she can pretend she read. We pay the compression tax in both directions and the bullet point that comes out is not the bullet point that went in. By 2025 the researchers had a name for the genre, workslop, and numbers to go with it: roughly forty percent of desk workers reported receiving it monthly, at a cost of about two hours of rework per incident. The chain is not a thought experiment. It is the org chart.

The chain, caught in the wild

Here is a real one from this spring, recreated with the names and numbers changed (the shape is exact). An engineer shipped a lifecycle analysis of a document pipeline: 48,217 submissions, a scary-looking 79% "failure" rate, and the forensics showing that almost none of it was real failure. Six tables deep, every claim traceable to a query. It is excellent work, and it costs twenty minutes to read properly.

#doc-pipeline · May 28 · the engineer

Headline funnel — 48,217 documents submitted (settled)

Funnel stageCount% of submittedNote
Rejected at gate (intentional)37,94478.7%healthy filtering: wrong path + duplicates
Real in-worker errors29<0.1%the only true failures
Status frozen (tracking bug)6,82014.1%≈4,433 actually completed
Completed-as-tracked3,4247.1%undercount; true completions higher

2b. Real in-worker errors — 29 (settled; the actionable failures)

Error bucketCountRoot cause
Worker exited prematurely: signal 911OOM kill; container exceeded memory mid-job
Got index error while flattening6code-level bug in PDF flattening
Unsupported file type … only PDF5non-PDF slipped past the gate
TimeLimitExceeded(55555,)4hard task time-limit (≈15.4h ceiling)
Image size exceeds limit3decompression bomb guard, working as intended

2c. Status frozen — 6,820 (tracking bug, NOT a processing stall)

Validated against the store: these records carry a stale status, but most of the documents actually processed. Breaking the bucket down by what the children did …

The engineer's report (recreated; names and numbers changed). It continues like this: six tables of forensics covering the frozen-status autopsy, the gate profile of the lost documents, time clustering, and provider mix. Excellent work, and twenty minutes of close reading.

Twenty minutes is more than most of the audience had, so a product manager did the sensible thing and asked a model to boil it down for the channel:

#doc-pipeline · May 28 · product, via the model

Summary: May 28 Pipeline Lifecycle Analysis

Overall picture

Of 48,217 documents submitted, the pipeline is healthier than the headline numbers suggest. The apparent ~79% "failure" rate is almost entirely intentional filtering, not breakage.

Funnel breakdown

  • Intentional gate rejections (37,944 — 79%): working as designed; wrong paths and duplicate documents filtered out. No action needed.
  • Genuine worker errors (29 — <0.1%): the only true processing failures: OOM kills, one flattening bug, a few oversized files.
  • "Frozen" status (6,820 — 14%): a tracking bug, not a processing failure; ~4,433 of these actually completed. A fix is in review.
  • The one real loss (1,154): silent webserver OOM under burst load, killing in-flight requests with no error and no retry. Recoverable via event-replay.

Key takeaway

The primary issue is observability, not throughput. The lost documents are recoverable, but the underlying cause needs to be addressed to prevent recurrence under burst load.

The re-summary (recreated). Nothing in it is wrong. Everything in it is the same size.

Nothing in the summary is wrong, and that is exactly what makes it instructive: this is the chain working as designed, and the signal is already going. Every fact arrives at the same volume, so the reader cannot tell the routine from the load-bearing. The one sentence that should change somebody's week (the loss will recur on the next traffic burst until the fix ships) sits mid-bullet, indistinguishable from the housekeeping. And this is hop one. The next hop keeps "pipeline is healthy" and drops the rest.

The old fixes knew something we forgot

None of this is a new problem, and the old solutions are instructive. Journalists built the inverted pyramid because telegraph-era editors cut stories from the bottom; the format was designed to survive truncation. The military codified BLUF, bottom line up front, into actual regulation. Minto taught consultants to argue in pyramids: answer at the top, evidence below, descend only as needed. And the scientific paper (title, abstract, introduction, body, appendix) is the oldest progressive TL;DR in production, shipping every altitude of the same idea in one artifact for four hundred years.

And the version I love most comes from my own trade: the C4 model. Simon Brown's whole pitch is that an architecture diagram should work like a map: you zoom. Context: here is the system and who touches it. Containers: here is what it is made of. Components: here is how one piece works inside. Code: here is the class, if you really must. I have reached for C4 for years, and it took watching the summary chain fail around me to understand why I trust it so much: it is choose-your-own-complexity. Nobody asks for a summary of a C4 set, because the summary is built in; the top diagram is the summary, and every deeper answer is one zoom away, drawn by the author from the same source of truth. A stakeholder and a new hire read the same artifact at different altitudes and neither one gets a guess.

Notice what all of them have in common: the detail stays attached. The reader chooses a depth and the rest of the document is right there, un-summarized, when they want more. The modern summary chain breaks exactly this property. Every hop strips the source and forwards only the compression. It is repeated truncation in which nobody keeps the original: the inverted pyramid with the bottom torn off at every desk it crosses.

Build the drill-down instead

So I stopped sending summaries and started sending altitudes. The artifact I now build has three fixed layers. The glance: the decision and the one caveat that matters, readable in ten seconds. The brief: the reasoning and the trade-offs, two minutes. The full: everything (the source document, the transcript, the raw numbers) attached, always. One artifact, written once, at the source, by the person with full context. The reader picks the depth. Nobody downstream re-summarizes, because the summary at every depth already exists and it was made by the one node in the chain that had the whole picture. It is C4 for prose.

For the pipeline report above, here is how those altitudes landed:

doc-pipeline · lifecycle health

28 May · settled · 48,217 submitted

The pipeline is healthy. The 79% "failure" rate is intentional filtering, not breakage. The one real problem: 1,154 documents silently lost to a webserver memory bug during traffic bursts. Recoverable.

Processing · healthy

29 / 48,217

genuine in-worker errors are negligible; the work that runs, runs clean

Observability · degraded

6,820

statuses mis-tracked; 4,433 (65%) show as stuck but actually finished (fix in review)

Intake · data loss

1,154

OOM-killed at intake during overnight bursts; recoverable via event-replay

reality check · what "79% failure" actually is

intentional reject (bad path + duplicate) · frozen, mostly finished · completed-as-tracked · real errors: 29

The artifact (recreated). The banner is the glance, the cards and the bar are the brief, and the engineer's full report rides along underneath as the full. Nobody downstream needs to re-summarize; every altitude already exists.

The honest objection is that this is more work, and it used to be true. Writing three altitudes of the same document was three writing jobs, which is why only journalists, scientists, and the military ever sustained the habit (they had editors, referees, and regulations to enforce it). That economics just changed. A model with the full source in context produces credible drafts of all three layers in about a minute. The author's job collapses to editing and verifying: one act of judgement at the source instead of five acts of guessing along the chain. The same tool that caused the slop makes the fix nearly free. We are just pointing it at the wrong end of the pipeline.

I am still tuning the format (the layer names change weekly, and the glance keeps wanting to grow a fourth sentence) but it has already paid for itself: the questions I get now are about the work, not about what I meant. The chain stops where the layers start.

Soapbox Who the hell has time to read that?
Filed under #Teams & Process#AI-Assisted Engineering#Opinion
Matthew Purdon

Matthew Purdon

Principal engineer and AI platform builder in Toronto with twenty-five years of experience turning messy software delivery into repeatable systems. Writes Field Notes on AI-assisted engineering and the new SDLC, and builds tools and processes in the Lab Reports.