Attributing Agent Cost and Latency to Work

What attribution means here

Attributing agent cost and latency to the work that caused it means tying every model call, its tokens, its price and its wall-clock time back to the specific task, feature or user that triggered it. It is the difference between knowing you spent $4,000 last month and knowing that $3,100 of it came from one retry-happy code-review agent.

The account-level number answers nothing

A provider’s billing dashboard gives you a single spend figure per account, sometimes split by API key. That figure cannot answer which feature, which task type, or which user caused a cost increase — which is the only question anyone actually asks when the number moves. Splitting by key helps only up to the point where one key serves several call sites, which is usually within a week of shipping. Once one agent runs several model calls per task and different task types share the same key, the account total is a smoke alarm with no room number: it tells you something changed and nothing about where.

It is a tracing problem, not a billing one

What attribution actually requires is an identifier that originates at the triggering unit of work — a ticket, a pull request, a user request, a scheduled job — and is propagated through every downstream model call, including calls made by tools the agent invokes and by sub-agents it spawns. That is context propagation, and it is a tracing concern. Billing exports arrive after the fact, aggregated, with no field you control; nothing you add at invoice time can recover a relationship the calls themselves never carried.

Distributed tracing already models this shape exactly. In OpenTelemetry’s trace model, a trace is one end-to-end operation and a span is one unit of work within it, with parent/child links and arbitrary key-value attributes. Mapped onto an agent: one trace per task, one span per model call, with token counts and computed cost recorded as span attributes. The generative-AI semantic conventions define the attribute names for this — model, operation, and input/output token usage — so that spans emitted by different services and SDKs aggregate without a per-service translation layer. Cost is not one of the conventional attributes; you compute it yourself from token counts and your own price table, which also means a price change is a table edit rather than a re-instrumentation.

The practical payoff is that the question “what did this feature cost” becomes a group-by over span attributes, and “why did this task cost so much” becomes reading one trace’s span list — usually a visible loop, a retry storm, or a tool call that fed 60k tokens of file contents back into context. The same propagation carries into routing decisions between models, because a router without per-task cost data is guessing about the thing it exists to optimise.

Latency and cost do not move together

Under agent workloads the two diverge, and an attribution model that tracks one as a proxy for the other will mislead you. A cheap model called twenty times in sequence can cost less than a single expensive call while taking considerably longer, because each call pays its own network round trip and its own time-to-first-token, and the twenty calls serialise if each depends on the previous one’s output. The reverse also holds: one large frontier call can be the cheapest wall-clock path to an answer while dominating the bill.

So record both, per span, and aggregate them separately. Span duration is native to the trace model; tokens and cost are attributes you attach. A task-level view then shows total cost, total wall-clock, and the sum of span durations — and the gap between the last two tells you how much of the work ran concurrently, which is the number that matters when you are running agents against provider rate limits and concurrency caps. Choosing which model handles which step is a separate decision, informed by this data rather than replaced by it; see frontier versus mid-tier selection per task class.

Cached input tokens are billed differently

An attribution model that counts raw input tokens and multiplies by one input price will misreport any caching-heavy workload — which describes most agents, since an agent re-sends a growing conversation and a stable system prompt on every turn. Prompt caching splits the input count into tokens read from a cache and tokens processed fresh, and those are priced differently. Anthropic’s prompt caching documentation describes the mechanism and the separate usage fields the API returns for cache creation and cache reads; check the current pricing page for the multipliers, since they are the input to your cost table and they change independently of the API.

Two consequences for instrumentation. First, capture the provider’s usage breakdown verbatim as span attributes rather than a single collapsed input number — you cannot reconstruct the split later, and a workload where most input is cache hits will be overstated by a naive model, sometimes by a large factor. Second, cache writes and cache reads are separate line items, so a workload that writes a cache entry and never reuses it costs more than the uncached equivalent; attribution that only counts reads will show caching as free savings when the arithmetic can go the other way for short-lived or highly varied prompts.

Caching also cuts latency independently of cost, which is another reason to keep the two measurements separate rather than deriving one from the other.

What to look up next

• The gen-AI semantic conventions are still evolving; check the current attribute names before writing a query that depends on them, and pin the convention version you emit.
• Your providers’ pricing pages, for the cache-write and cache-read multipliers that feed the cost table.
• How your agent runtime propagates context across process and queue boundaries — a trace that breaks at the queue produces orphaned spans and unattributed spend. This is closely tied to durable execution for long-running agent workflows, where the run outlives a single process by design.
• Retention and sampling policy for traces: agent traces are large, and head-based sampling will silently drop the expensive outliers you most want to see.