Cloudflare published an engineering breakdown of how the company reduced the average cost of a single AI code review from $2.43 to $0.77, a 68% reduction, and cut the median review time from 4 minutes 09 seconds to 1 minute 32 seconds. The main source of overpayment was not in pricing tiers, but in the mechanics of prompt caching: the first request of each session almost always wrote to the cache instead of reading from it. By fixing the cacheable prefix, switching Anthropic models, and reducing unnecessary calls, the team simultaneously made reviews cheaper and faster without losing quality.
What happened
Cloudflare engineer Ryan Skidmore published a step-by-step breakdown of the optimization of the internal AI reviewer—an agent that automatically checks code changes before merging. Starting point: in early August, all reviewers switched to Anthropic models, after which the cost of a single job rose from $1.38 to $2.03, and token consumption per review increased from 0.91 million to 2.99 million, a 3.3x increase. The first and main cause of overpayment was found in the cache configuration: OpenCode v1 placed a single cache breakpoint at the end of a block that mixed static prompts with dynamic repository data, so the first request of each session almost always paid for cache writes instead of reads—hits occurred only 14–26% of the time. After separating the block and moving the common prefix between repositories, the hit rate for first requests rose to 91–99.5%. Another ~60% of savings came from switching models and effort levels: Opus 5.5 instead of Opus 5 and Sonnet 5.5 instead of Sonnet 5 were 47–62% cheaper per run at the same token price, while reducing unnecessary LLM calls lowered, for example, the average number of calls for code quality checks from 6.2 to 1.7 on Sonnet 5.5 medium. Result: the average cost of a single review dropped 3.2x to $0.77, and the median job runtime dropped 2.7x to 1 minute 32 seconds.
Context
To understand where the savings came from, it is useful to recall the prompt caching mechanics of providers like Anthropic: reading a cached prefix costs ten times less than regular tokens (0.1x), its initial write is a quarter more expensive (1.25x), and the cache lives for only 5 minutes. From this follows a non-obvious practical conclusion: the order of data in the prompt is a monetary issue. If changing fields like date, working directory, or tool list end up in the middle of the cacheable block, the prefix no longer matches, and each session pays for an expensive cache write instead of a cheap read. The second layer of context is the trajectory of agentic pipeline development: the Cloudflare reviewer had been expanding functionality for several months, and token consumption was growing faster than cost control; this is a typical ailment of maturing agentic systems, not the result of a single error. The third layer is measurement methodology: in addition to cost and latency, the team tracked the average number of important findings per review and the share of findings for which the author actually changed the code; these were the metrics used to ensure the optimization did not break the tool's purpose.
Why this matters for the industry
For the industry, the case is valuable because it is a rare public breakdown of the unit economics of an agentic pipeline with specific numbers, confirmed by working production, not a demo: savings were achieved simultaneously with an increase in the average number of important findings from 0.28 to 0.30 and an increase in the share of findings where the author actually changed code from 34% to 59%. The main takeaway for teams operating LLM agents: the inference bill is not a fixed tax on model prices, but a manageable engineering discipline. The structure of the cacheable prefix, routing between models and effort levels, and reducing the number of calls affect margins just as much as the choice of the model itself—while the main overpayments were in the configuration, not the price list. The signal for agentic startups is that unit economics is becoming a product metric on par with quality and speed, and observability such as first-request hit rate, tokens per job, and cost per unit of useful work is a candidate for standard elements of agentic pipeline dashboards.
Why this matters for users
For owners of their own agentic systems, the case provides a checklist that can be verified using token logs for one day and does not require a new deployment. First, ensure that the first request of each session reads from the cache, not writes to it: a low hit rate on first requests usually indicates a poorly placed or broken cache breakpoint. Next, it makes sense to move dynamic data—date, working directory, tool list—out of the cacheable prefix, leaving the static part of the prompt at the front. Then, simpler techniques work: reading files in parallel in the agent's first response, skipping extra sub-reviewers on repeat reviews, and stopping the process immediately after publishing the result instead of idling through the task. A separate lesson is about model updates: when switching versions at the same token price, savings came from the new model spending fewer tokens on the task—this is an argument to compare versions by actual token consumption on your workload, not just by price.
What is still unknown / limitations
Caveats are significant. Quality metrics in the case were collected by the same team that performed the optimization: an average of 0.30 important findings versus 0.28 may be within the noise, the share of findings where the author changed code is also self-reported, and there is no independent validation of these estimates. Hit rate ranges and savings amounts were obtained on a specific Cloudflare configuration, so on other pipelines, the techniques are more likely to be reproducible than the exact numbers. Finally, the cache pricing mechanics—expensive write, cheap read, 5-minute TTL—reflect Anthropic's pricing at the time of the post and may change.
Sources
Author
Look at AI, editorial team