Section 01 · Definition
What cost per completed task actually measures
Token counts are a cost proxy, not a cost unit. The right unit ties spend to the business outcome the agent was hired to produce.
Quick answer
In one sentence: Cost per completed task is the total spend — tokens from every model call, all tool-call fees, retry tokens from failed branches, and human review time — divided by the number of tasks that reached a completed state.
Most teams instrument tokens. Tokens are the easiest thing to pull from provider APIs and dashboards, so they become the default proxy for cost. The problem is that tokens measure consumption, not output. An agent that retries four times before succeeding and one that succeeds in a single pass have wildly different token counts but produce the same business result. The four-retry agent is four times more expensive. Tokens alone hide that entirely.
Cost per task surfaces what tokens obscure: the full economics of producing a usable result. It also creates a benchmark you can compare across models, frameworks, and agent designs. When you migrate from GPT-4o to a smaller model, the question is not whether tokens per call dropped — it is whether cost per completed task dropped. Retries matter. Failure rates matter. Tool call counts matter.
Section 02 · Cost components
The four costs that make up every completed task
Each component is real and measurable. Leaving any one out understates the true cost and misleads the design decisions you make from the number.
Token cost — all model calls, not just the first one
Sum tokens across every model call the agent makes during the task: planning calls, tool-dispatch decisions, tool-result interpretation, reflection passes, and final output generation. Use provider APIs (not dashboard estimates) for billed token counts. On most tasks, this is 50 to 80 percent of total cost.
Tool call fees — compute, search, storage, third-party APIs
Tool calls have their own cost lines independent of the language model. A web search call through a retrieval API costs $0.005 to $0.01 per call. A code execution sandbox charges per second of compute. A vector similarity search charges per query. In retrieval-heavy workflows, tool costs can exceed model costs.
Retry overhead — the cost of every failed branch
When an agent fails and retries, every token in the failed branch is real spend. A task that requires three attempts at a complex reasoning step costs three times the tokens for that step. Track retry rate alongside cost — if your retry rate exceeds 15 percent, retries are likely your biggest per-task cost driver.
Human review time — approval gates and sampled audits
If your human in the loop pattern includes a pre-execution gate, an exception escalation, or a sampled audit, the reviewer's time has a cost. At $75 per hour (a conservative internal rate for a senior reviewer), a two-minute review adds $2.50 to the task cost. At $25 per hour, it is still $0.83. For high-volume workflows, human review often dominates.
Section 03 · Instrumentation
How to capture the metric with OpenTelemetry spans
Provider dashboards are a starting point, not a complete picture. You need spans that wrap the full task lifecycle, not individual model calls.
The right instrumentation boundary is the task, not the model call. Create an OpenTelemetry span when a task starts and close it when the task reaches a terminal state — completed, failed after max retries, or escalated to a human. Every model call, tool invocation, and retry inside the task is a child span. Attach cost metadata to each child span at the point of execution.
For each model call child span, attach: the model ID, input token count, output token count, billed cost from the provider SDK, and whether the call was a retry. For each tool call child span, attach: tool name, call duration, billed cost if available, and success or failure. When the parent task span closes, aggregate cost across all children.
Most LLM observability platforms — Langfuse, Helicone, Portkey, Arize — read OpenTelemetry spans and display cost breakdowns per task. You can use any of them or build your own aggregation on top of raw spans pushed to an OTEL collector. The platform choice matters less than the span boundary choice. Span the task, not the model call.
The retry-attribution rule
Failed model calls that trigger a retry generate real token spend but produce no task value. Attribute that cost to the task regardless — it is part of the true cost. Do not exclude retries from cost reporting. Excluding them makes the metric optimistic and hides the most actionable improvement signal: high retry rates usually mean a prompt design or model selection problem, not a cost problem.
Section 04 · Budget tiers
Reasonable cost-per-task targets by workflow complexity
These ranges come from production agent deployments. Use them as design constraints, not targets to optimize toward.
Workflow complexity drives cost more than model choice. The biggest driver of high per-task cost is not the model — it is the number of tool calls, the retry rate, and whether a human review step fires. Set your budget tiers first, then design the workflow to fit inside them.
| Workflow tier | Examples | Target range | Primary cost driver |
|---|---|---|---|
| Lightweight | Classify, route, summarize, tag, translate | < $0.02 | Token cost (1–3 calls) |
| Medium | Research, draft, compare, extract | $0.02–$0.25 | Tool calls + retries |
| Complex | Multi-step audit, write-then-verify, regulated review | $0.25–$2.00 | Retries + human review |
| High stakes | Financial transaction review, legal document analysis | > $2.00 acceptable | Human review dominates |
When a task lands outside its tier band, treat that as a signal. A lightweight classify task costing $0.15 usually means an unnecessary tool call, an over-specified prompt that generates extra output tokens, or a retry loop triggered by an ambiguous input. Debug the workflow, not just the cost.
Section 05 · Multi-agent attribution
How to attribute cost across agents in a chain
In a multiagent system, attributing cost to a single call or a single agent is not enough. You need cost ownership at each hand-off boundary.
A multiagent chain typically has an orchestrator and one or more worker agents. The orchestrator dispatches sub-tasks, the workers execute them and return results. Each worker is responsible for its own sub-task cost — the tokens it spent planning, the tool calls it made, and the retries it triggered. The orchestrator is responsible for its own model calls plus the coordination overhead.
The attribution boundary is the hand-off: the point where the orchestrator sends a sub-task to a worker and waits for a result. Open an OpenTelemetry span at that boundary keyed to the worker's agent ID. Everything the worker spends before returning its result accumulates inside that span. When the hand-off closes, you know exactly what that worker cost for that sub-task.
This matters more than it sounds. Without attribution at the hand-off boundary, a multiagent system looks like a single cost blob. You cannot tell whether the orchestrator or a specific worker is responsible for a cost spike. With attribution, you can identify the expensive agent in the chain and redesign that one component — smaller model, fewer tool calls, tighter retry budget — without touching the rest of the chain.
Attribute to the task, not the session
A session may contain multiple tasks (or a single long-running task that spans many turns). Attribution at the session level hides per-task economics and makes it impossible to compare task designs. Always attribute to the completed task as the primary unit.
Track cost concentration across agents
In production, 20 percent of agents typically generate 70 to 80 percent of cost. Run a periodic report: cost per agent per task type. The expensive agent is almost never the orchestrator — it is usually a retrieval-heavy worker with unoptimized search patterns or no result caching.
Section 06 · Enforcement
Where spend caps actually work — and where they don't
A budget that lives in a system prompt is not a budget. A budget enforced by the orchestrator before a tool call is a control.
The most common mistake in agent cost control is adding a spend instruction to the system prompt: “Try to minimize tool calls and keep the cost of this task under $0.50.” This does nothing. The model cannot see its own accumulated cost mid-task. It has no mechanism to enforce a cap. Under load, it ignores the instruction entirely.
The right pattern is to enforce spend caps in the orchestrator's policy engine or in the tool dispatch layer. Before each tool call, the orchestrator checks the running accumulated cost for the current task span. If that cost has reached the per-task budget, the orchestrator either routes to a cheaper fallback tool, downgrades the model for the remaining steps, or flags the task for human review instead of continuing. That check happens before the tool call — not after the money is already spent.
Set caps at three levels: per tool call (to catch individual expensive operations), per task (to bound the total), and per session or hour (to bound aggregate spend in burst scenarios). The per-task cap is the most important. The per-tool cap catches outliers. The session cap is a safety net for runaway loops.
If you are evaluating an agent system for production, the AI Systems Architecture service covers cost instrumentation as a first-class concern alongside observability, evaluation, and scale. Cost-per-task benchmarking is one of the first layers we wire in on any new agent system — it gives you a baseline before you optimize anything and a comparison point after every change.