Four token buckets, not two

Most LLM observability tools record two token counts per request: input and output. Anthropic bills four. That gap is not a rounding error — it is the difference between a cost figure you can act on and one that is quietly wrong.

The four buckets are:

anthropic.usage.input_tokens            ordinary prompt tokens, input rate
anthropic.usage.output_tokens           generated tokens, INCLUDING extended thinking
anthropic.usage.cache_creation_tokens   written into the prompt cache — premium rate
anthropic.usage.cache_read_tokens       served from the prompt cache — deep discount

Cache creation costs more than ordinary input. Cache reads cost dramatically less. A tool that reports only two numbers has to fold both cache buckets into "input", and the moment it does, the cost it reports stops tracking the invoice.

What the collapse actually does to a number

Take a shape from our conformance fixtures — an agent turn with a large, stable system prompt and a small amount of new context:

• 500 fresh input tokens
• 220 output tokens
• 40,000 cache reads

A two-bucket tool reports 40,500 input tokens. That is arithmetically true and completely useless. Nearly all of those tokens were served from cache at a steep discount, but the number presented to you is indistinguishable from 40,500 tokens of fresh prompt — which would cost roughly an order of magnitude more.

Run that error across a month of agent traffic and your cost dashboard is not slightly off. It is telling you the opposite of what is happening: the workloads that look most expensive are frequently the ones caching best.

The number you actually want

Four buckets give you the cache-hit ratio, which is the single most actionable efficiency figure available on an LLM workload:

cache_read / (input + cache_creation + cache_read)

For the example above, that is 0.988. That one number tells you prompt caching is working. Its absence is why so many teams have a vague sense that caching "probably helps" without ever being able to demonstrate it.

It is also the lever that matters commercially. Storage is not where the margin on an LLM product lives; cache efficiency is. A team that cannot measure its cache-hit ratio cannot optimise the thing that most affects its unit economics.

The arithmetic trap that will bite you

There is a fifth attribute, and it is the most likely source of a cost bug in any system consuming this data:

anthropic.thinking.tokens

Extended-thinking tokens are a subset of output_tokens, not a fifth bucket. Add them to a total and you double-count them, inflating every figure downstream — cost per request, cost per customer, cost per eval run.

total = input + output + cache_creation + cache_read     // thinking excluded

Our reference implementation excludes it deliberately, and a test enforces that exclusion, because it is exactly the kind of error that looks plausible in a code review and silently corrupts a quarter of cost reporting.

Why cost is computed at ingest

One more decision that follows from taking cost seriously: we calculate cost when a span arrives, against a versioned price book, and store the result alongside the version that produced it.

Computing at read time seems simpler and is a trap. It means historical costs silently change whenever prices change — so a report you ran in March stops matching the same report run in June, with no record of why. Storing the computed value with its price-book version is what makes a three-month-old cost figure reproducible instead of merely plausible.

---

The four buckets are part of the SouthBase span specification, which is an extension of OpenInference rather than a proprietary format — so the data stays portable. The full attribute list is in the span specification.