Why an AI Agent Costs 30x a Chatbot, and How to Fix It

Teams that have run a chatbot for a year and then ship their first agent get a surprising invoice. The model did not change price. The usage pattern did, and agents consume tokens in a way that breaks the intuitions built on request-response chat.

The multiplier is real and it is large

Measured across production deployments, agents consume 5 to 30 times the tokens of a comparable chatbot interaction. A 2026 arXiv study of coding-agent tasks found the extreme end of that distribution: agentic tasks using up to 1000 times the tokens of a code chat exchange, with as much as 30x variance on the same task depending on the path the agent took.

That variance is the part worth sitting with. The same task, run twice, can differ thirtyfold in cost. Any budget built on an average is a budget built on a number the system does not respect.

Where the tokens actually go

The cause is architectural rather than mysterious. An agent sends its full accumulated context, including the system prompt and everything that has happened so far, to the model at every step. By step 20 of a multi-step task, the agent has paid for the earliest part of that context twenty times.

Concretely: an agent working with a 25,000-token context that takes five refinement steps sends 125,000+ input tokens for a single job. Nothing has gone wrong. That is the design working as intended. Real workflows land somewhere between 19 and 50 times the cost of a single model call, depending on scope and how much context each step drags along.

The lever nobody pulls

That table contains the most useful thing in this article. Because agents resend the same prefix by design, agent workloads are the most cache-friendly workloads that exist. And cache-hit pricing across the major providers is aggressive in a way that base rates are not.

This reverses a conclusion you would reach from list prices alone. DeepSeek raised its rates sharply in August, and for one-shot calls that makes it a worse deal than it was. For agentic work with a stable prompt prefix, a 30x cache discount on the repeated portion matters more than the base rate, and V4-Pro remains one of the cheapest places to run a long agent loop.

Getting hits requires discipline the framework will not enforce for you. Keep the system prompt and tool definitions byte-identical between steps, put anything that varies at the end of the context rather than the start, and stop regenerating prompts with timestamps or session IDs near the top. A single changed character at the front of the prefix invalidates the cache for everything after it, and most teams discover this by watching a bill rather than by reading a doc.

Which model behind which tier

Agent frameworks encourage pointing one capable model at every step. That is the single most expensive default in the category, because the steps are not equally hard. Routing by step type is where the large savings sit.

• Routing, classification, and tool selection go to the cheapest capable model. Gemma 4 26B A4B at $0.070 / $0.300 handles deciding which tool to call, and this is often half the steps in a loop.
• Extraction and structured output go to the same tier. Constrained output shape means quality differences do not surface, and output is where pricing hurts most.
• Planning and multi-step reasoning justify a stronger model. DeepSeek V4-Flash scores 52 on the Artificial Analysis Intelligence Index at max effort, and the cache economics on V4-Pro suit long loops.
• Final synthesis, where a user reads the output, is worth paying for. This is usually one call in twenty, so a frontier model here costs little and is the step a customer actually judges.
• Nothing goes to a reasoning mode it does not need. Reasoning tokens bill as output tokens, and output is three to five times input on most providers.

What agents actually cost to run

Production figures for business agents span $400 a month at the low end for narrow, well-scoped workflows, to $7,500 a month or more for complex multi-function agents serving real users at scale. The distance between those numbers is mostly architecture, not model pricing: how much context each step carries, how many steps run, whether the cache is being hit, and whether a cheap step is being sent to an expensive model.

Which is the honest summary of this whole topic. Switching provider changes your bill by some percentage. Fixing context discipline, step count, and routing changes it by a multiple. Before you migrate anything, read why per-token price predicts your bill poorly, because the same measurement discipline applies and the agent case is where it pays best.

Three things to check this week

Log tokens per completed task, not per call, and log the distribution rather than the mean. Given 30x variance on identical tasks, the mean will mislead you and the 95th percentile is what determines whether a busy week breaks the budget.

Check your cache hit rate. If your provider reports cache-hit tokens separately and the number is near zero, something at the front of your prompt is changing between steps, and fixing it is usually an afternoon of work for a large recurring saving.

Count how many steps in your loop genuinely need the model you are paying for. In the audits our AI and automation team runs, the answer is typically fewer than half, and the routing fix pays for itself faster than any migration.