TokenAtlasToken pricing, in one paragraph
Every modern LLM is billed per token. A token is roughly four characters or three-quarters of a word in English. Providers publish two prices per model — input ($/1M tokens) and output ($/1M tokens) — and the output price is typically 3–5× higher because generation is sequential and GPU-bound. The cost of a single request is (input_tokens × input_price + output_tokens × output_price) / 1,000,000. Sounds simple; in production it isn't.
Why AI bills surprise teams
Three reasons keep recurring. First, output tokens dominate the bill once you ship features that reason or write, but most engineers estimate from prompt size. Second, retrieval-augmented generation silently injects thousands of tokens of context on every call. Third, agentic workflows multiply tokens by the number of tool-calling rounds — a five-step agent costs roughly five times a single chat turn at the same model. Forecast in expected tokens per session, not requests per second.
The real economics of inference
Frontier models (GPT-class, Claude Sonnet/Opus, Gemini Pro) price for reasoning quality. Mid-tier models (GPT-mini, Claude Haiku, Gemini Flash) price for throughput. Open-weights hosted endpoints (Llama, DeepSeek, Mistral) price for floor cost. The right architectural move is rarely 'pick the cheapest model' — it's routing: cheap models for the 80% of low-complexity calls, frontier models for the 20% that need them. In our own pricing models, routing this way lands well below a single-model baseline — run your own token mix through the calculator to see the modelled difference for your workload.
RAG, embeddings, and the cost of memory
Embeddings are cheap to compute and cheap to store, but every retrieval call costs LLM input tokens. A 10K-document corpus with 2K-token chunks and top-5 retrieval injects roughly 10K input tokens per query — often more than the user's prompt. Vector databases add a small per-query fee on top. The lever is chunk size and retrieval depth, not the embedding model.
Unit economics for AI products
A defensible AI product needs healthy gross margin at steady-state usage. Calculate it as (price − COGS) / price, where COGS = average tokens per user-month × blended model price + vector storage + supporting infrastructure. If the margin is thin, the fix is almost always model routing, prompt compression, or response caching — not negotiating with the provider.