Caching Strategies: Prompt Caching & Context Caching

Repeated prompt prefixes cost money and time on every single call. Prompt caching stores them between calls and saves both.

The problem: the same text, over and over

An agent often sends the same long opening with every request: system prompt, tool definitions, large documents. Without caching, the model recomputes this part completely on every single call — that costs time (latency) and money (token price), even though the text didn't change.

How prompt caching works

Anthropic and OpenAI both offer prompt caching for this. The provider remembers the beginning (prefix) of a prompt you already sent once. If you send that same beginning again shortly after — e.g. the same system prompt plus a new user turn — the model only has to process the new part. The rest comes from the cache.

What it saves

With Claude, a cache hit costs only a fraction of the normal price for input tokens, while a cache write is somewhat more expensive than normal. With OpenAI, prompt caching is automatically active once a prompt is long enough, and it noticeably lowers cost and latency, with no code changes required.

TTL: the cache expires

A cache doesn't live forever. With Claude, the default is 5 minutes; after that the entry is gone, and the next call writes a fresh cache entry. There's also a longer, more expensive option. Whoever sends many requests in quick succession benefits the most from caching.

EXAMPLE

An agent with a 6,000-token system prompt plus tool definitions makes 10 requests in 3 minutes. Without caching, you pay the full price for those 6,000 tokens on all 10 calls. With caching, you pay it once (write, 1.25x) and only 0.1x for the remaining 9 — a clear saving, as long as the requests stay within the 5-minute TTL.

🛠️ EXERCISE — TRY IT YOURSELF

Find out whether and how one of your agent scripts (or your model provider's API docs) uses prompt caching.

Look at your agent's system prompt / tool definitions: roughly how many tokens does this fixed part have?
Check the API response (if you use Claude) for the fields cache_read_input_tokens and cache_creation_input_tokens — is the cache being hit?
Think about it: with which usage pattern (many requests in quick succession vs. single rare requests) does caching pay off most for you?

✅ SELF-CHECK

☐ Can you explain why a cache hit is cheaper than a cache write?
☐ Do you know how long a default Claude cache lives before it expires?
☐ Have you identified a concrete part of your own prompt that would be a good fit for a cache breakpoint?

QUICK QUIZ

Why does a prompt cache entry expire on its own?