Controlling Agent Costs Without Making the Product Worse
A system-level approach to AI agent cost using routing, caching, compact tools, cancellation, budgets, and outcome-based measurement.
Agent cost is the sum of every model turn, tool call, retry, search, and abandoned workflow—not the advertised token price of one model. Cost control therefore begins with architecture and product behavior.
Measure cost per completed outcome
Attach usage to a task identifier and record whether the job completed, needed correction, or was abandoned. A workflow that looks cheap per turn can be expensive per success if it loops or escalates often.
- Route routine steps to the least expensive model that passes the eval
- Cache stable instructions, retrieved context, and deterministic tool results
- Summarize large tool outputs before they re-enter the loop
- Set turn, time, and spend budgets with graceful stop behavior
- Cancel work when the user leaves or the business outcome is no longer possible
Remove repeated uncertainty
If the agent repeatedly rediscovers the same rules, encode them in tool schemas, workflow state, and deterministic validation. Models should spend reasoning on the part that changes, not on reconstructing stable application logic.
Protect the user experience
Do not hide budgets behind silent truncation or degraded answers. If a limit is reached, preserve completed work, explain what remains, and offer a clear way to continue with the right level of intelligence.
Primary sources
First-party documentation and announcements used to ground this field note.
