Love this framing, good piece - it has always been about the outcome. I think the novelty of agents has caused some to forget that what we should always have in mind is the finished product. Once prices got real folks started allocating less time to playing and more time to how to get to a price efficient outcome.
I keep seeing clients optimise the visible number too — the cheapest term plan premium, the lowest expense ratio fund — while the real cost sits in the gap between what they hold and what the goal actually needs.
A lower premium that leaves a coverage shortfall, or a smaller SIP that forces a top-up scramble later, is the same error as a shorter tool response that triggers two recovery turns.
Measuring the wrong unit feels like progress right up until the finished job comes due.
This is the classic ops trap. You squeeze the visible tokens, then the agent runs three extra commands to recover what you hid, and the bill goes up. GitHub's result matches every incident where we "optimised" the wrong meter. So IMO, cut the retries and the thrash first. Short prompts are not the same as cheap behaviour.
The shell-output example is a useful warning about measuring one response instead of the whole task. I'd compare total tokens and elapsed time through completion. A missing line can send the agent back for another command, and that next call carries the growing history too.
The GitHub result is a clean example of optimising the visible number. A shorter response that forces a recovery turn costs more than the longer one, because every extra turn drags the whole history along. The same logic holds one level up: in a build the expensive thing is rarely the tokens, it is the review round that sends the work back. The work you never have to redo is the only kind of saving that holds up. Do the teams you talk to track retries separately from first attempts?
the line numbers anecdote is the one i'd pin to the wall. nobody's prompt review ever catches formatting that used to matter, and that kind of dead weight compounds silently. most of 'context rot' is probably just archaeology nobody scheduled.
Token efficiency in agentic workflows is the hidden tax on security operations. When agents consume massive context windows to solve simple tasks, the operational cost eventually eclipses the value of the automation itself. We see this friction when models attempt to bypass security guardrails by iterating through complex, token-heavy evasion paths. My recent look at how agents adapt their behavior when facing restrictions shows that forcing shorter, more precise interaction loops actually improves the resilience of the system. It turns out that constraining the input space forces the model to find the most direct path to the objective.
There is a hard floor under this that the vendor documentation states plainly. Anthropic's prompt caching page sets a minimum cacheable prompt length of 512 tokens for Claude Opus 5.5, Fable 5.1 and Mythos 5.1, 1,024 tokens for Sonnet 5, and 4,096 tokens for Haiku 4.5. Below that, a request marked with cache_control is processed without caching and no error is returned. Cache writes cost 25 percent more than base input tokens on the five-minute TTL. A prompt trimmed under the floor therefore pays full input price on every call, quietly, and the invoice is the only place it appears.
ruben, the bit about measuring cost per finished job instead of per tool call is the line i'll steal. the rust token killer result is so counterintuitive it almost has to be true: cheaper calls, more expensive tasks, because recovery turns drag the whole accumulated context with them.
the detail that stuck with me: github's shipped compressor leaves diffs and file contents alone. one missing line sends the agent back, and your compression costs more than it saves. there's a general rule hiding in there: never compress the thing the agent can't recover cheaply.
also the automated prompt rewrite silently serialising parallel subagents. compression deletes behaviour, not just tokens.
The part that stuck with me is the prompt rewrite that quietly turned a soft suggestion about parallel subagents into a strict rule, and it passed offline evals and nobody caught it till users did. thats the real risk with any compression pass, it deletes behavior you never wrote a test for. Makes me think system prompts need to be treated more like code, if a line matters it should have an eval behind it before anyone trims it. Also agree cost per finished task is the right unit, per call numbers look great on a dashboard and tell you almost nothing about what the job actually cost.
Love this framing, good piece - it has always been about the outcome. I think the novelty of agents has caused some to forget that what we should always have in mind is the finished product. Once prices got real folks started allocating less time to playing and more time to how to get to a price efficient outcome.
I keep seeing clients optimise the visible number too — the cheapest term plan premium, the lowest expense ratio fund — while the real cost sits in the gap between what they hold and what the goal actually needs.
A lower premium that leaves a coverage shortfall, or a smaller SIP that forces a top-up scramble later, is the same error as a shorter tool response that triggers two recovery turns.
Measuring the wrong unit feels like progress right up until the finished job comes due.
This is the classic ops trap. You squeeze the visible tokens, then the agent runs three extra commands to recover what you hid, and the bill goes up. GitHub's result matches every incident where we "optimised" the wrong meter. So IMO, cut the retries and the thrash first. Short prompts are not the same as cheap behaviour.
The shell-output example is a useful warning about measuring one response instead of the whole task. I'd compare total tokens and elapsed time through completion. A missing line can send the agent back for another command, and that next call carries the growing history too.
I also wrote a tutorial on context management, now at 1k+ GitHub stars: https://github.com/hardness1020/learn-agent-architecture/tree/main/sections/08-context-management
The GitHub result is a clean example of optimising the visible number. A shorter response that forces a recovery turn costs more than the longer one, because every extra turn drags the whole history along. The same logic holds one level up: in a build the expensive thing is rarely the tokens, it is the review round that sends the work back. The work you never have to redo is the only kind of saving that holds up. Do the teams you talk to track retries separately from first attempts?
the line numbers anecdote is the one i'd pin to the wall. nobody's prompt review ever catches formatting that used to matter, and that kind of dead weight compounds silently. most of 'context rot' is probably just archaeology nobody scheduled.
Token efficiency in agentic workflows is the hidden tax on security operations. When agents consume massive context windows to solve simple tasks, the operational cost eventually eclipses the value of the automation itself. We see this friction when models attempt to bypass security guardrails by iterating through complex, token-heavy evasion paths. My recent look at how agents adapt their behavior when facing restrictions shows that forcing shorter, more precise interaction loops actually improves the resilience of the system. It turns out that constraining the input space forces the model to find the most direct path to the objective.
https://cyrilsimonnet.substack.com/p/openais-agents-were-told-no-they?utm_source=substor&utm_medium=substack&utm_campaign=comment
There is a hard floor under this that the vendor documentation states plainly. Anthropic's prompt caching page sets a minimum cacheable prompt length of 512 tokens for Claude Opus 5.5, Fable 5.1 and Mythos 5.1, 1,024 tokens for Sonnet 5, and 4,096 tokens for Haiku 4.5. Below that, a request marked with cache_control is processed without caching and no error is returned. Cache writes cost 25 percent more than base input tokens on the five-minute TTL. A prompt trimmed under the floor therefore pays full input price on every call, quietly, and the invoice is the only place it appears.
ruben, the bit about measuring cost per finished job instead of per tool call is the line i'll steal. the rust token killer result is so counterintuitive it almost has to be true: cheaper calls, more expensive tasks, because recovery turns drag the whole accumulated context with them.
the detail that stuck with me: github's shipped compressor leaves diffs and file contents alone. one missing line sends the agent back, and your compression costs more than it saves. there's a general rule hiding in there: never compress the thing the agent can't recover cheaply.
also the automated prompt rewrite silently serialising parallel subagents. compression deletes behaviour, not just tokens.
The part that stuck with me is the prompt rewrite that quietly turned a soft suggestion about parallel subagents into a strict rule, and it passed offline evals and nobody caught it till users did. thats the real risk with any compression pass, it deletes behavior you never wrote a test for. Makes me think system prompts need to be treated more like code, if a line matters it should have an eval behind it before anyone trims it. Also agree cost per finished task is the right unit, per call numbers look great on a dashboard and tell you almost nothing about what the job actually cost.