OpenAI released a prompt caching update for the GPT-6 family: cache boundaries and time-to-live have become explicit API parameters, and the discount on cached input tokens reaches 90%. For products with large repeating prefixes — agentic systems, RAG pipelines, and coding assistants — this is a direct way to cut API costs.

image
image

What happened

The update affected the entire cache lifecycle. Cache hit rate metrics have improved; explicit stop points (explicit breakpoints) are set via the prompt_cache_options.mode and prompt_cache_breakpoint parameters, and the cache time-to-live for GPT-5.6 and newer is configured via the prompt_cache_options.ttl parameter — for example, the value "30m" means 30 minutes since the last cache write or use. The official API documentation specifies a minimum cacheable prefix of 1,024 tokens and up to 4 cache writes per request. Under the new pricing, writing to the cache costs 1.25x the regular input token price, and reading from the cache — 0.1x. The update also includes the Prompt Cache Diagnostics tool for analyzing misses, the Prompt Caching Dashboard, and cache warming before a user request. OpenAI also cites a GitHub benchmark: the share of tokens requiring fresh processing dropped by more than 50% across billions of requests.

Context

The mechanics of prompt caching — reusing already-computed KV states when the beginning of a request matches: if a new call has the same prefix, the model does not recalculate it. The technique itself has been known for a long time and carries almost no scientific novelty; what matters is different. Previously, caching was a hidden server-side optimization whose effectiveness depended on random prefix matches; now cache boundaries, TTL, and diagnostics are part of an explicit API contract. Breakpoints solve the main practical problem of prefix caching: they allow explicitly separating the immutable part of the prompt from the dynamic tail. The economics can be calculated in advance: two requests with the same prefix cost 1.35x versus 2x without caching, and ten requests — 2.15x versus 10x.

Why this matters for the industry

The cache has become a first-class part of the API contract, and this changes product design, not just bills. Builders of long-context agents — coding agents, research systems, RAG services — can explicitly set cache boundaries and change reasoning effort between responses without resetting it, i.e., separate reasoning quality from request price. Persistent agents with a large stable context get predictable unit economics instead of hoping for a lucky prefix match. For startups, this is primarily a pricing shift, not a platform shift: the economics of already-working products improve, rather than requiring a rewrite for a new platform.

Why this matters for users

If you work with the OpenAI API, the effect is available immediately but requires active work: check the current hit rate on the Prompt Caching Dashboard (platform.openai.com/usage?usage_section=prompt-caching), analyze misses via Prompt Cache Diagnostics, isolate a stable prefix of at least 1,024 tokens in your prompts, place explicit prompt_cache_breakpoint if needed, and tune prompt_cache_options.ttl to your request rhythm. For repeating prompts, input tokens can cost up to 10% of the regular price, and the first token of the response arrives faster; for persistent sessions, it is useful to configure cache warming according to the request schedule.

What is still unknown / limitations

The only published quantitative result — the vendor GitHub benchmark — is not accompanied by disclosed methodology: no dataset, load mix, baseline definition, or significance statistics, and the mention of billions of requests is an argument of scale, not method. Until there is an independent reproduction, this figure should be considered at the marketing level of evidence. Real savings depend on your hit rate and prompt structure, so they need to be measured on the dashboard, not taken from the announcement. The expectation that explicit cache controls — breakpoints, TTL, diagnostics, warming — will become a standard part of API contracts at other major providers is an interpretation, not an established fact.

Sources

Author

Look at AI, editorial team