TokenAtlasWhy OpenAI API costs balloon
Token cost = (input tokens + output tokens) × per-model rate. Teams overspend because they default to GPT-4o for everything, resend the same system prompt on every call, and never inspect where the tokens actually go. A single agent loop that appends full history each turn can quadruple monthly spend within a week.
1. Right-size the model per route
GPT-4o-mini is $0.15/$0.60 per 1M tokens vs GPT-4o at $2.50/$10 — a 16× spread. Route classification, extraction, formatting, and simple chat to mini; reserve GPT-4o for reasoning-heavy work. Most teams cut 40–60% by splitting one route.
2. Turn on prompt caching
OpenAI bills cached input at 50% off. Identical prefixes ≥1024 tokens auto-cache. Move system prompts, tool schemas, and few-shot examples into a stable prefix — a 4K-token system prompt at 100K calls/month goes from $1,000 to $500 for that segment alone.
3. Use the Batch API for anything async
Batch jobs run at 50% off input and output, completing within 24h. Evals, embeddings backfills, classification sweeps, and nightly summarization belong on Batch. This is the single easiest 50% cut for offline workloads.
4. Cap max_tokens aggressively
Output tokens cost 4× input on GPT-4o. Uncapped completions routinely burn 800+ tokens on tasks that need 120. Set max_tokens per route based on p95 real usage, not model defaults.
5. Trim context, don''t append it
Naive agent loops resend full history each turn — a 20-turn conversation at 500 tokens/turn quietly grows to 100K+ tokens by the end. Summarize older turns, drop tool outputs after use, and keep a rolling window sized to real recall needs.
6. Compress system prompts
A 2,000-token system prompt on 100K daily calls is $7,500/month in input alone on GPT-4o. Delete politeness, redundant examples, and unused tool definitions. Most system prompts halve without measurable quality loss.
7. Switch embeddings to text-embedding-3-small
text-embedding-3-small is $0.02/1M tokens — 5× cheaper than 3-large and within 2% of MTEB quality for most retrieval workloads. Migrate unless you''ve benchmarked and confirmed you need the larger model.
8. Instrument spend per route, not per model
Aggregating by model tells you nothing about which feature to fix. Log token counts tagged with route/feature and view the breakdown daily. The most expensive route is almost never the one your team thinks it is.
9. Set hard budget alerts
OpenAI''s usage limits are project-wide and lagging. Wire per-route soft caps in your gateway that alert at 50%, 80%, and 100% of daily budget. This catches runaway agent loops within minutes instead of at end-of-month.
Worked example: cutting a $12K/mo bill
Support agent, 100K calls/day, 4K in / 600 out on GPT-4o = $11,700/mo. Route classification to mini (−45%), cache the 3K-token system prompt (−22% on remaining), cap output at 400 tokens (−15% on remaining). New bill: ~$3,900/mo. Same feature, same latency.
Frequently asked questions
- What are current OpenAI API costs?
- As of 2026: GPT-4o is $2.50 input / $10 output per 1M tokens. GPT-4o-mini is $0.15 / $0.60. GPT-4 Turbo is $10 / $30. Cached input runs at 50% off; Batch API runs at 50% off both directions.
- How is OpenAI token cost calculated?
- Cost = (input tokens × input rate + output tokens × output rate) per 1M tokens, per model. A tokenizer estimate is roughly 1 token per 4 English characters; use TokenAtlas for exact per-model estimates before shipping.
- What''s the fastest way to cut OpenAI spend?
- Route lower-complexity work to GPT-4o-mini and enable prompt caching on stable system prompts. Together these commonly cut 50–70% of bill within a week, no quality drop on routes that don''t need flagship reasoning.
- Does prompt caching cost extra?
- No — cached input is billed at 50% of the standard rate with no cache-write fee on OpenAI. Prefixes ≥1024 tokens cache automatically when identical across requests.
- How does TokenAtlas help?
- TokenAtlas estimates per-model cost before you ship a route, tracks live spend by feature, benchmarks equivalent models across providers, and surfaces the specific caching, batching, and model-swap wins for your workload.