Section 01 · Definition
What agentic AI cost optimisation actually means
A single LLM call is easy to reason about. An agent is harder, and the place where most of the money goes is not where most teams look first.
Quick answer
What is agentic AI cost optimisation? Agentic AI cost optimisation is the set of techniques that reduces the token spend of multistep agent systems by compressing context, caching repeated work, routing to the right model per step, grounding with retrieval, and pruning state, while holding output quality constant.
A single LLM call is easy to reason about. You send a prompt, you pay for input plus output tokens, you get a response. An agent is harder. An agent calls the model many times to complete one task, every step usually carries the full conversation plus tool outputs plus retrieved context, and that bundle grows with each step. The cost per task is the sum of every call, not the cost of one. If you optimise the agent the way you would optimise a chat endpoint, you miss the place where most of the money is going.
The first thing to internalise is that agentic systems spend most of their tokens on the same content sent repeatedly. Cutting that repetition is where the largest, lowest risk savings live.
Section 02 · Root Cause
Why agents spend so much: context explosion
A user request that started at 200 tokens becomes a 10,000 token request by the third step and 50,000 tokens by the end of a moderate task.
The pattern is consistent across production agents. A user request that started at 200 tokens becomes a 10,000 token request by the third reasoning step and a 50,000 token request by the time the agent finishes a moderate task. The growth comes from a few predictable places.
System prompts get long because they have to encode tool descriptions, formatting rules, refusal policies, and few shot examples. That entire block rides along with every step. Tool outputs are verbose by default. A function that returns a JSON document, a database row set, or a search result can add thousands of tokens that the agent only needed one or two values from. Conversation history is sticky. Every step usually includes the prior messages so the agent can reason about what it has already tried. Retrieved snippets compound. A retrieval call brings back five documents of 500 tokens each, and those snippets stay in the context for the rest of the reasoning loop unless something prunes them.
Multiply that by the number of reasoning steps. A simple task might run for three steps. A medium task runs ten to fifteen. A long running agent on a multi document task can run thirty or more. Even with cheap input pricing, the bill compounds quickly because each step pays for the cumulative context, not just the new content.
The diagnosis most teams miss
If the cost surprises you in production, the surprise is almost always context growth. Direct token cost per call is rarely the problem. Decompose the spend by step count and input tokens before you change anything else.
Section 03 · Levers
The five cost levers, ranked by impact and risk
Each lever has a different effort to impact ratio. Treat them as a prioritised stack rather than a menu.
Semantic caching
Cache prompt response pairs at a semantic similarity threshold. The first time an agent answers a question, you pay for the call. The next time a similar question arrives, you serve the cached response. For workflows with repeated user intents, semantic caching alone can cut twenty to forty percent of spend with almost no quality risk.
Model routing
Not every step needs the most expensive model. Classification, intent detection, format conversion, and simple tool calling can run on a smaller, cheaper model. Reasoning and synthesis run on the larger one. A simple router that sends seventy percent of calls to a cheap model and thirty percent to the expensive one often delivers forty to sixty percent of total savings on its own.
Context and prompt compaction
Most system prompts can be shortened thirty to fifty percent without quality loss once you measure carefully. Tool descriptions can be compressed. Few shot examples can often be dropped after the first few interactions. Compacted history, where older turns are summarised instead of carried verbatim, is one of the highest impact levers on long running agents.
Retrieval grounding instead of context dumping
Many agents are built to stuff documents into the prompt and let the model find what it needs. A grounded retrieval pattern flips that around. You retrieve a smaller set of relevant passages and tell the model what to use. The hit on context size is usually three to five times smaller and the answers are often more accurate.
Memory pruning and state hygiene
Agents that carry state across runs accumulate noise. Old tool outputs, stale plans, and resolved subtasks add tokens without adding value. A periodic pruning step, scoped to the agent memory model, recovers context budget. High reward on long running agents and irrelevant on simple ones.
A useful mental model: the first two levers are mostly mechanical and benefit from off the shelf tooling. The next three need product judgement and an evaluation harness to confirm quality holds.
Section 04 · Sequencing
The right order of operations
The temptation when costs spike is to attack the most visible problem. In practice the impact ranking and the effort ranking do not match.
A safer sequence is caching plus routing first, then compaction, then retrieval grounding, then memory pruning. The reason is risk asymmetry. Caching and routing have minimal quality risk when configured carefully, and they each deliver step change savings. Get those in place and most agent budgets are already in a healthy range. Only then is it worth investing in compaction work that requires careful evaluation, because the marginal savings are smaller and the quality risk is higher.
This sequencing also lets you measure each lever independently. If you bundle them into one big release, you have no idea which one moved cost down and which one quietly hurt quality. Land them one at a time, run an eval suite after each, and you keep the cost curve and the quality curve visible at the same time.
If you are working with an outside team on this, the right deliverable is not a tools recommendation. It is a measured rollout plan that pairs each lever with a quality eval and a target savings number. The agentic AI consulting engagement is built around exactly this kind of sequenced optimisation.
Section 05 · Measurement
What to measure as you optimise
Cost optimisation without observability turns into guesswork. Three measurements matter most.
Cost per task is the headline number. Pick a unit of business value, like a resolved support ticket or a generated report, and track total token spend per unit. This is the metric a finance team will care about and the only one that ties spend to value.
Quality per task is the constraint. Pick an evaluation rubric and run it on a stable sample of tasks before and after each lever change. If quality drops below a defined threshold, roll the change back. If quality is unchanged, ship and move to the next lever. The LLM observability guide covers the specific signals worth wiring in.
Cost decomposition is the diagnostic. Break the spend into input tokens, output tokens, and step count. Most teams discover their problem is input token growth driven by accumulated context, not the model price they thought it was. Until you can see the split, you cannot tell which lever to pull.
The weekly review that keeps the work defensible
A simple weekly review of cost per task, quality per task, and cost decomposition, plotted next to each release, makes the optimisation work visible. Without that, the engineering team gets accused of either over engineering or under delivering, and neither is true.
Section 06 · Limits
When optimisation is the wrong fix
There is a point where the levers run out. If the unit economics still do not work after the easy levers, the problem is the architecture.
Common signs the problem is architectural: the agent is doing too many things and would be better split into two narrower agents, the task scope is unbounded and any optimisation just moves the bill rather than fixing it, or the model is being asked to reason over data that should have been pre processed in a deterministic pipeline first. In each case, more optimisation work delivers smaller and smaller returns while the right answer is to redesign the system. See the agentic AI production architecture guide for what a redesign typically looks like.
The honest framing for stakeholders is that cost optimisation is a phase, not a permanent state. The first wave of work, the five levers in order, typically delivers forty to seventy percent savings inside one to two months. After that, the curve flattens. If you are still chasing five percent savings six months later, the original architecture probably needed to change.
For a deeper treatment of the single call cost curve that an agent inherits, the LLM inference cost optimisation playbook covers the per call levers this post stacks on top of.
Section 07 · FAQ
Frequently asked questions
How much can agentic AI cost optimisation actually save?
A well sequenced rollout of the five levers (caching, routing, compaction, retrieval grounding, memory pruning) typically delivers forty to seventy percent token spend reduction inside one to two months without hurting output quality. Caching plus model routing alone usually account for the first forty to sixty percent of the savings.
Why are AI agents so expensive to run at scale?
An agent calls the model many times per task, and every call carries the cumulative context: system prompt, prior turns, tool outputs, and retrieved snippets. A 200 token user request becomes a 10,000 token request by step three and a 50,000 token request by step ten. The bill is paid on the cumulative context, not just the new content.
What is context explosion in AI agents?
Context explosion is the growth of an agent input across reasoning steps. System prompts, tool descriptions, conversation history, and retrieved snippets accumulate, and each step pays for the full bundle. It is the single largest driver of agent cost and almost always the right place to look first when bills surprise you.
Should I use semantic caching or model routing first?
Implement both, but if you can only land one this sprint, start with model routing. It almost always delivers the larger single change (forty to sixty percent of total savings) and the implementation is mechanical. Semantic caching adds another twenty to forty percent on workflows with repeated intents and pairs naturally with routing.
When is cost optimisation the wrong fix for an expensive agent?
If you have implemented caching, routing, compaction, retrieval grounding, and memory pruning, and the unit economics still do not work, the problem is the architecture. Common signs: the agent scope is too broad, the task is unbounded, or the model is reasoning over data that should have been preprocessed in a deterministic pipeline. Redesign beats further optimisation.