An AI price card might say $5 input / $30 output per million tokens. That does not mean a request costs $35. It means each part of the request is metered separately, and the expensive part depends on how much text the system processes, generates, repeats, reasons over, and sends to tools.
Token pricing looks like a software unit. Underneath it are accelerators, memory bandwidth, batching, networks, buildings, electricity, engineering, and provider margin. This guide explains the meter without requiring you to become an infrastructure accountant.
The output is one-fifth the token volume but more than half the bill.
What is a token?
A token is a chunk produced by a model-specific tokenizer. It can be a word, part of a word, punctuation, whitespace, code fragment, or byte pattern. One English word is not always one token. Other languages, unusual identifiers, tables, and encoded data can tokenize very differently.
Do not budget from page count alone. Use the tokenizer or usage field for the actual model. The same text can produce different counts after a provider changes model family.
Input tokens can include far more than what a user typed:
system and developer instructions;
conversation history;
retrieved documents;
tool descriptions and schemas;
images or audio converted into model units;
application metadata;
examples and formatting constraints.
Output can include visible text, structured JSON, code, tool arguments, and sometimes hidden reasoning tokens.
Why output costs more
During input processing, hardware can calculate many prompt positions in parallel. During generation, each next token depends on the tokens before it. The model repeatedly loads state and computes one step, occupying scarce serving capacity until it stops.
Providers therefore price output throughput higher. They also price commercially, not just physically: generation is where customers perceive value, high output reserves capacity, and different service tiers carry different margins.
The operational lesson is simple. A concise answer limit can save more than trimming a few prompt sentences. Ask for the format you need, not “everything you know.”
Four meters hidden behind “input” and “output”
Uncached input
New tokens processed at the standard input rate.
Cached input
Repeated prefixes eligible for a discount. OpenAI and other providers publish separate cached-input rates for some models. A cache hit is usually based on exact or provider-defined prefix matching; similar meaning is not enough.
Cache writes
Some providers charge extra to create a cache, with different prices for short and long retention. Anthropic, for example, documents write multipliers and discounted cache reads. Caching pays only when the prefix is reused enough times before expiration.
Output and reasoning
Visible output is billed at the output rate. Reasoning models may consume additional internal tokens. The usage object—not the visible character count—is authoritative.
Context caching with worked math
Suppose 1,000 daily requests share a 40,000-token handbook and add 2,000 unique tokens. Each returns 1,000 output tokens. Assume:
With a successful cache after the first write, approximately:
text
40M cached tokens → $20
2M new tokens → $10
1M output tokens → $30
plus cache-write/retention cost
daily total before cache write ≈ $60
Caching can cut this workload by roughly 75%. If the handbook changes every request, traffic is low, or prefixes do not match, the saving disappears.
Our context-window pricing guide explains why an advertised one-million-token window can also trigger long-context price tiers and repeated-turn multiplication.
Agents multiply the meter
A chat request may be one model call. An agent task can be ten:
Interpret task.
Plan.
Search.
Read results.
Call a database.
Inspect output.
Retry a failed tool.
Draft.
Verify.
Revise.
If the full conversation and tool results are resent each time, input grows turn by turn. A 20,000-token starting context across eight calls is not 20,000 billable tokens. It can exceed 160,000 even before appended observations.
Web search, code execution, file processing, image generation, embeddings, speech, and managed agent features may have separate meters. A search can be priced per call while retrieved text also becomes input tokens. A code sandbox may charge by session time. Vector databases charge storage and queries.
Read the price card for both model usage and built-in tools. A cheap model that makes twenty paid searches can cost more than a better model making three.
Batch, priority, and service tiers
Providers may discount asynchronous batch work because it can be scheduled when capacity is available. Priority or low-latency service may cost more. Reserved throughput trades commitment for capacity certainty.
Classify workloads:
Interactive: latency matters; standard or priority.
Background: hours are acceptable; batch.
Predictable high volume: compare reservations.
Spiky experiments: pay-as-you-go.
Do not pay interactive rates for an overnight classification job.
Long-context tiers
Some providers charge higher rates after a prompt crosses a threshold. A model with a one-million-token capacity may price inputs above 272,000 tokens differently, and the higher multiplier can apply to the full session rather than only tokens above the line.
This creates a cost cliff. Retrieving 250,000 relevant tokens may be much cheaper than sending 300,000 poorly filtered tokens, even before quality degradation. Capacity is a ceiling, not a target.
Subscription price versus API tokens
A $20 consumer subscription is not usually a bag of API credits. It bundles interface features and access subject to limits, fair-use controls, model availability, and changing policies. API use is separately metered.
For a professional doing interactive research and writing, a subscription can be cheaper and easier. For automation, integration, observability, and repeatability, API pricing applies. Compare them with our white-collar AI subscription guide.
Five ways bills surprise teams
1. Conversation history grows
A long chat resends earlier messages. Start a new thread or summarize state when old detail is unnecessary.
2. RAG retrieves too much
More documents can lower answer quality while raising input cost. Tune chunking, ranking, and top-k against citation accuracy.
3. Outputs are unbounded
Set useful maximums and schemas. Do not ask for ten alternatives if two enter the workflow.
4. Retries are invisible
SDKs, agents, or application logic may retry rate limits, invalid JSON, and tool failures. Attribute every call to the original task.
5. Reasoning is hidden from the UI
Visible output can be short while reasoning usage is large. Log provider usage fields and model tier.
Build a cost dashboard that matters
Track:
Metric
Reason
Input, cached input, output, reasoning
Explains the invoice
Calls and tool calls per task
Exposes loops
Cache hit rate
Validates caching economics
Cost per accepted task
Connects spend to value
P50/P95 cost
Finds long-tail blowups
Human repair time
Prevents shifting cost to labor
Model and prompt version
Explains changes
Customer/workflow allocation
Enables budgets and pricing
Set a per-task cap and a monthly alert. An agent should stop or request approval before a retry cascade spends more than the task is worth.
A procurement example
Suppose 50 employees each complete 20 AI-assisted tasks per workday, 22 days a month. Each accepted task averages 30,000 uncached input, 10,000 cached input, and 3,000 output tokens on a $5/$0.50/$30 price card.
That is about $0.245 per task before tools and software. If the workflow saves five minutes of a $40/hour employee's time, gross labor value is $3.33 per task. If review takes six additional minutes, the economics reverse. Token cost is only useful beside outcome and labor.
Bottom line
Input pricing pays for what the model reads. Output pricing pays a higher rate for what it generates. Caching discounts repeated prefixes. Reasoning and tools can add meters you do not see in the answer. Agents repeat the entire equation across turns.
Budget at the task level, log the provider's usage object, and divide by accepted outcomes. Once you do that, a price card stops being marketing fog and becomes an engineering input.
A five-minute bill audit
Export one day of usage and choose ten representative tasks. For each task, group every model call under a single task ID. Add uncached input, cached input, output, reasoning, tool calls, and retries. Compare the provider usage record with your application log; a missing retry or tool call is a measurement bug.
Next, sort tasks by cost and inspect the top ten percent. Long tails usually reveal one of four problems: conversation history that never resets, retrieval returning too many passages, unbounded output, or a loop retrying without changing strategy. Fix that cause before negotiating a lower token rate.
Finally, attach an outcome. Mark whether the task was accepted and record human repair minutes. A workflow that costs $0.08 but consumes twelve minutes of repair is not cheaper than one that costs $0.40 and is ready in two minutes. This small audit converts a provider invoice into a product decision and gives finance, engineering, and users the same denominator.
Prices and billing categories vary by provider and change frequently. The formulas are evergreen; verify current model, cache, long-context, reasoning, tool, batch, and regional rates before committing spend.
