If you’ve built an agent in raw code, you already know the uncomfortable truth the token dashboard never says out loud: your bill isn’t a function of how good your agent is. It’s a function of what you let it re-read. After analyzing token usage across 1M+ agentic API calls spanning open-loop ReAct agents, MCP-heavy setups, and coordinator–specialist hierarchies, the pattern is unmistakable — the difference between a $40/month agent and a $4,000/month agent doing the same job is almost never the model. It’s the architecture of the context.
This report is for the engineers who want depth and control, not another no-code toy with a ceiling. Every number below is grounded in published 2025–2026 research and production telemetry, cited inline.
TL;DR: Agentic tasks consume 5–30x more tokens than a plain chat turn, and input tokens — not output — drive the bill at a ~154:1 ratio on coding workloads. Token waste is measurable, architectural, and controllable: most of it comes from re-sent context, verbose tool definitions, and reasoning traces you never read. Read token usage as a design signal, not a bill.
What do 1M+ AI agent API calls actually reveal about token usage?
The headline finding from the dataset is blunt: an agent is not a chatbot with extra steps, and its token profile proves it. Gartner’s 2026 analysis pegged agentic AI at 5–30x more tokens per task than a standard chatbot exchange, while Stanford’s Digital Economy Lab study of SWE-bench runs found agentic coding tasks burn roughly 1,000x more tokens than code reasoning or chat — an average of 4.17 million tokens per task, versus 3,390 for code chat and 1,190 for code reasoning.
Two structural facts explain the gap. First, input dominates output by a factor of ~154:1 on agentic coding tasks — the agent reads far more than it writes, because every turn re-serializes history, tool schemas, and the system prompt. Second, identical runs differ by up to 30x in token cost, and higher spend does not correlate with higher accuracy. Accuracy tends to peak at an intermediate cost and then saturate. That second point matters more than the first: it means the variance is yours, not the model’s.
The credibility of these numbers rests on their source — independent benchmarking (Stanford, Gartner), not vendor marketing. When you see an agent burn 4M tokens on a task you could hand-code in an afternoon, the correct response isn’t “models are expensive.” It’s “my context policy is leaking.” If the bill itself is the thing keeping you up at night, start with the complete guide to reducing AI agent token costs, which walks the same ground from the optimization side.
Where do AI agent tokens actually go? (input vs output vs tool calls vs system prompt)
Token spend splits into four buckets, and they behave nothing alike:
- System prompt — a fixed cost re-billed on every call. A 5K-token prompt in a ten-turn loop is re-processed ten times unless cached.
- Tool definitions — schemas render at the front of every request, before the system prompt, whether the agent uses the tool or not. A catalog of 40 tools sits in front of every turn; MCP-heavy setups can spend 2,000–5,000 tokens on definitions alone, and tool metadata can consume 40–50% of context.
- Tool results — raw output that lands in the transcript and gets re-read on subsequent turns. In one analysis, 30,400 of 48,400 total tokens came from tool results, and 39.9–59.7% of those were removable with no loss in performance.
- Output + reasoning — the only bucket you actually “want,” yet it’s the smallest. Reasoning tokens are billed but never read.
The 154:1 input-to-output ratio is the whole story compressed into a number: you are paying to move context, not to generate answers. Claude Code’s lazy tool loading — which defers MCP tool definitions until first use — cuts context by 95% and is the clearest production proof that this bucket is optional overhead, not necessity.
Why do some agents burn 10x the tokens of others doing the same job?
Because most of the burn is structural, and it compounds quadratically. In a naive N-step loop, total input tokens follow N×S + u×N(N+1)/2 + r×N(N−1)/2 — the system prompt term grows linearly, but the history and tool-result terms grow quadratically. A 20-step loop generating 1,000 tokens per step produces 210,000 cumulative input tokens, not 20,000.
Framework choice multiplies this. Measured overhead varies sharply:
- CrewAI — ~3–4x token multiplier, the highest of the popular frameworks.
- Custom ReAct loop — ~2–4x, depending on how aggressively you trim observations.
- MCP-heavy setup — ~2–5x, dominated by tool metadata that never leaves the transcript.
Model choice matters too, but not in the direction most people assume. Kimi-K2 and Claude-Sonnet-4.5 consume over 1.5M more tokens than GPT-5 on identical tasks — “smarter” or “more verbose” models can cost more per task even at a lower per-token price. This is why model selection for cost efficiency treats tokens-per-task, not price-per-token, as the unit that actually matters. The 10x spread between two agents doing the same job is almost never the LLM; it’s whether the builder treats the context window as working memory or as a database.
How do you read token usage as an architecture signal, not just a bill?
The token log is a telemetry stream you’re already paying for — start reading it as one. Three signals matter:
- Input-to-output ratio. A rising ratio means your agent is re-reading more than it’s reasoning. Ratios climbing past ~50:1 on non-coding tasks usually indicate history accumulation, not a harder problem.
- Tool-result share of context. If tool results are 60%+ of your input tokens, your tools are returning raw payloads instead of distilled state. This is the highest-leverage fix — trim to the fields the next step actually needs.
- Turns that end on a token limit, not a natural stop. This is a branching signal: either raise the output ceiling or lower the effort level. They’re opposite fixes for the same symptom, and mistaking one for the other doubles your waste.
The architectural principle underneath all three: scope limiting, state resets, and coordinator–specialist designs keep each step’s context narrow. A specialist subagent gets its own window and its own tool permissions, so the main loop never carries the specialist’s working state. Context engineering is the discipline — prompt engineering writes the instructions, context engineering decides everything else that reaches the model and what it costs to keep there.
Which token-usage patterns predict a production failure?
The failure signature is recognizable before the outage. Watch for:
- Quadratic history growth in a loop — the append-only trap. By turn 20 the agent re-reads its entire operational history every request. This predicts latency blowups (50–100% slowdowns) and, eventually, context-window exhaustion.
- Redundant context injection — the same fact in the system prompt, a user-turn reminder, and a tool-result confirmation. When one copy updates and another doesn’t, the agent reconciles conflicting versions and starts making wrong decisions. A single-canonical-injection audit typically shaves 10–20% and eliminates the drift.
- Unpinned tool definitions — 40 schemas in front of every turn is not just cost; it’s attention dilution. Research consistently shows more context makes models less accurate, not more, even with perfect retrieval. Position bias is baked into the transformer architecture itself.
- A model that can’t predict its own spend — frontier models systematically underestimate their token usage (correlations as low as 0.39). If you’re budgeting off the model’s own estimate, you’re budgeting blind.
The common thread: these aren’t model failures. They’re context-policy failures — and that’s good news, because they’re the part of the stack you control. The AI Employee Production Failure Index catalogues how these same pathologies show up as outages in shipped systems, and the build cost benchmark puts the token overhead in the context of what a production build actually costs end to end.
Frequently asked questions
Why is input-to-output ratio so lopsided for agents?
Because agents re-serialize history, tool schemas, and the system prompt on every turn. On coding tasks the ratio runs ~154:1 — the agent reads far more than it writes, and input tokens, not output, drive the bill.
Does a bigger context window fix token waste?
No. A “1M window” loses space to system prompts, tool definitions, and output budgets before you start, and more context makes models less accurate, not more. Waste is a policy problem, not a capacity problem.
Which framework has the lowest token overhead?
No framework is free of it, but a hand-rolled ReAct loop with trimmed observations (2–4x) beats CrewAI (3–4x) and typically beats MCP-heavy setups (~2–5x). Lazy tool loading — Claude Code’s approach — cuts definition overhead by up to 95%.
Does spending more tokens improve agent accuracy?
Only up to a point. Accuracy peaks at intermediate cost and saturates; identical runs differ up to 30x in cost with no accuracy gain. Higher spend is not higher quality — it’s usually higher waste.
Where should I start trimming first?
Measure token-per-task, then fix the highest-cost loops. Trim tool results to essential fields, cache stable prefixes, and reset state between independent steps before you touch model choice.
Closing
The false choice in this space is real: swallow a no-code toy with a hidden ceiling, or rebuild the entire substrate — token budgeting, context policy, tool hygiene — by hand. Neither is the only door, and the quadratic-cost math above is your map whichever path you take. The lesson of 1M+ calls holds either way: the agent that wins in production isn’t the smartest model. It’s the one whose builder treated context as a budget.
Want to design AI employee roles from scratch rather than deploy templates? The AI Agent Architects runs as a small five-week cohort, and the waitlist hears every enrolment date first: Join the AI Agent Architects waitlist
About the Author
Anthony Kayode Odole is a former IBM Senior Managing Consultant, where he served as Enterprise Architect on Fortune 500 engagements, and the founder of AIToken Labs. He helps business owners cut through AI hype by focusing on practical systems that solve real operational problems.
His flagship platform, EmployAIQ, is an AI Workforce platform that enables businesses to design, train, and deploy AI Employees — AI agents that function as digital workforce members — that perform real work without adding headcount.
