Prompt caching is a discount on text you send again and again. Many AI applications send the same long text with every question: instructions, a price list, a returns policy. With prompt caching, the provider keeps its processed version for a short time. A request that starts with exactly the same text reuses it: that part costs about a tenth of the normal price, and the answer starts sooner.
How does it work?
A language model turns every token of the input into intermediate results, the keys and values of the KV cache. Prompt caching stores those results for the start of a prompt, the prefix, so a later request with the same prefix skips that work (OpenAI). Only an exact match counts. One changed character early in the prompt, such as a timestamp, makes everything after it new again. So put the fixed parts first (tool definitions, the system prompt, reference documents) and the question last. The answer itself is the same as without caching (Anthropic).
At OpenAI caching is on by default. At Anthropic you switch it on with a cache_control field in the request, either once for the whole request or on up to four blocks that change at different rates. If you run a model on your own server, vLLM does the same with automatic prefix caching.
What does it cost?
In September 2026 both providers use the same multipliers: writing a prefix to the cache costs 1.25 times the normal input price, and reading it costs 0.1 times, for most models at Anthropic and for GPT-5.6 and later at OpenAI. For Claude Sonnet 5, which costs $2 per million input tokens, a cache write costs $2.50 and a cache read $0.20 per million tokens. Output tokens cost the same with or without a cache.
Caching pays off from the first reuse. In OpenAI's words, writing a prefix once and reusing it once costs 1.35 times its normal input price, against 2 times for processing it twice without a cache. Anthropic's cache lasts five minutes by default and is renewed free of charge each time it is read; a one-hour cache costs twice the input price to write. For GPT-5.6 and later, OpenAI keeps a prefix for at least 30 minutes after its last use.
How much faster is it?
The gain grows with the length of the cached text. When Anthropic launched prompt caching in August 2024, it measured the time until the answer began: for a 100,000-token prompt (a whole book) it fell from 11.5 to 2.4 seconds, for a 10,000-token prompt from 1.6 to 1.1 seconds.
Where does it go wrong?
- Short prompts are not cached. In September 2026 the minimum is 1,024 tokens for Claude Sonnet 5 and for OpenAI's GPT-5.6, and 4,096 for Claude Haiku 4.5. Below that, Anthropic processes the request normally without an error.
- A quiet spell empties the cache. At Anthropic, after five minutes without a request, the next one pays the write price again.
- Parallel requests miss each other. Anthropic makes a cache entry available only once the first response has begun.
- Caches are separate per organisation at both providers, so other customers cannot read yours.
Example
Suppose a webshop's customer service assistant sends 10,000 tokens of instructions and product information with every question, plus about 500 tokens of question. It handles 2,000 questions a day, one every 20 seconds or so during opening hours, so the cache stays warm. On Claude Sonnet 5 at September 2026 prices, that is 21 million input tokens a day without caching: $42. With caching, the 20 million cached tokens cost $4 and the 1 million question tokens $2, so $6 a day, plus about 2 cents each time the cache has to be written again.