The cache line

A prompt cache is a prefix match, and almost every way of getting it wrong looks exactly like getting it right.

July 2026

Prompt caching is the rare optimisation that is both enormous and nearly free: one field, in the right place, and the largest part of your prompt costs a tenth as much. It is also the rare optimisation that fails silently. There is no error, no warning, and no field in the response that says "your cache never once hit." There is only the invoice, arriving three weeks later, being wrong in a way nobody can explain.

I spent an afternoon last spring convinced our caching was working because the code said cache_control in it. It was working the way a smoke alarm with no battery is working.

Suppose nrequests per day5,000 requests a day hit the same Sonnet 5 prompt, and the frozen part of that prompt — pstable prefix tokens12k tokens of system prompt and tool definitions — never changes. Uncached, you pay $7,380 a month. With one cache_control breakpoint in the right place, you pay $2,521 — a 2.93× difference, or $4,859 a month you were setting on fire.

Holding fixed: 900 volatile tokens and 700 output tokens per call, traffic spread over 10 active hours, 5-minute TTL, 30-day month, list prices.

The mechanism is worth understanding properly rather than by cargo cult, because every failure mode falls out of one sentence: a prompt cache is a prefix match on bytes. Everything else — the TTLs, the multipliers, the minimum lengths, the four-breakpoint limit — is detail hanging off that sentence.

Three blocks, one order

Before anything is cached, the request is assembled. Three regionsThe order is fixed by the API and is not configurable. Reordering your own code changes nothing about it., always in this order: tools, then system, then messages. The result is hashed as a byte string, and the hash is the cache key. noteFour cache_control breakpoints per request, maximum. In practice one, at the end of the stable region, does almost all the work — a second only earns its place when you have two genuinely different reuse lifetimes in one prompt.

That gives you the only rule you actually need: stable content first, volatile content last. Not "stable content marked with a breakpoint" — stable content physically earlier in the rendered prompt. A marker cannot protect bytes that sit above something that changes.

tools = sorted(TOOL_DEFS, key=lambda t: t["name"])

system = [{"type": "text", "text": FROZEN_PROMPT}]

system[-1]["cache_control"] = {"type": "ephemeral"}

messages = [{"role": "user", "content": f"{now()}\n{question}"}]

The write is an option

A cache write costs more than not caching. That is the part people skip, and it is the part that makes the arithmetic interesting rather than obvious.

So you are buying an option. You pay a small premium up front for the right to re-read those tokens cheaply, and the option expires — five minutes by default, an hour if you pay double for the write. What matters is how many times you exercise it before it lapses.

And the option can expire worthless. A hit resets the TTL, so a warm entry survives indefinitely on its own traffic — but if calls arrive further apart than the TTL, every single one finds an expired entry, and caching becomes a pure surcharge with no upside. noteWhich is the honest argument for the one-hour TTL. It is not "better"; it is the same bet with a longer expiry, priced accordingly. Take it when your traffic has gaps of five to sixty minutes, and not otherwise. The chart below crosses over for exactly this reason: at the left-hand end, not caching wins.

Across a day of realistic traffic the difference is not subtle: the effective price of the same prefix tokens tracks under a working cache and under a poisoned one — flat, because a poisoned prefix never amortises anything.

How a prefix gets poisoned

Nothing so far is hard. The hard part is that the failure is invisible, and every plausible-looking piece of code below causes it.

The tell is always the same, and it is one field: usage.cache_read_input_tokens. If it is zero across repeated requests that should share a prefix, something above your breakpoint is moving. noteDo not read input_tokens as "the prompt size". It is the uncached remainder only. Total prompt size is input_tokens + cache_creation_input_tokens + cache_read_input_tokens, and an agent that ran for an hour showing 4k input_tokens is a working cache, not a small prompt.

SYSTEM = load_frozen_prompt()  # no interpolation, ever

resp = client.messages.create(
  model="claude-sonnet-5",
  system=[{"type": "text", "text": SYSTEM,
           "cache_control": {"type": "ephemeral"}}],
  tools=sorted(TOOLS, key=lambda t: t["name"]),
  messages=[
      *history,
      {"role": "user",
       # Volatile context goes here, below the breakpoint.
       "content": f"Today is {date.today()}.\n\n{question}"},
  ],
  max_tokens=16000,
)
assert resp.usage.cache_read_input_tokens > 0  # the only real check

The list of things that do this is short and boring, which is what makes it easy to miss in review: a date or timestamp in the system prompt; a UUID or request id; json.dumps without sort_keys=True; iterating a set; a per-user tool list; conditional system sections, where every combination of flags is a distinct prefix; and — the one that gets everybody — switching models mid-conversation, because caches are model-scoped. noteThe pattern that saves you here is architectural rather than clever: freeze the system prompt as a constant at import time and make interpolating into it impossible rather than merely unwise. If a value has to reach the model mid-conversation, put it in the messages array, not in the system field.

The floor nobody mentions

There is one more way to have a perfectly correct, perfectly stable, perfectly marked prefix that never caches: it is too short.

Below the model's minimum, the breakpoint is accepted, the request succeeds, and cache_creation_input_tokens comes back as 0 forever. And the minimum is not monotonic across the range — the newest model has the lowest floor, so a prefix that caches happily on one model can go inert when you route the same traffic to a cheaper one. is the Opus 5 floor drawn against the Haiku 4.5 floor.

Do the arithmetic yourself

Everything above is one pure function with about forty lines in it. Here it is with the sliders attached — the same function that drew every chart, table and sparkline on this page.

What I actually changed

Three things, none of them clever.

I moved the timestamp out of the system prompt and into the user turn — the single highest-value four-line diff I have ever written, worth $4,859$4,859 a month at our volume. I froze the system prompt as a module-level constant so that interpolating into it requires editing the constant, which requires a review. And I added one assertion to the smoke test: cache_read_input_tokens > 0. That assertion has caught the regression twice since.

The last one is the real lesson. Prompt caching does not need monitoring or dashboards or a cost anomaly detector. It needs one boolean, checked once, in a test that already runs — because the only thing that makes this failure expensive is that nothing was watching for it.

Thanks to everyone who has ever asked me “but why is the bill like that,” which is the question this whole page is an answer to. Mistakes, rounding and simplifying assumptions are mine; the prices are Anthropic’s published list rates and will drift.