TokenAtlasToken pricing, in one paragraph
Every modern LLM is billed per token. A token is roughly four characters or three-quarters of a word in English. Providers publish two prices per model — input ($/1M tokens) and output ($/1M tokens) — and the output price is typically 3–5× higher because generation is sequential and GPU-bound. The cost of a single request is (input_tokens × input_price + output_tokens × output_price) / 1,000,000. Sounds simple; in production it isn't.
Why AI bills surprise teams
Three reasons keep recurring. First, output tokens dominate the bill once you ship features that reason or write, but most engineers estimate from prompt size. Second, retrieval-augmented generation silently injects thousands of tokens of context on every call. Third, agentic workflows multiply tokens by the number of tool-calling rounds — a five-step agent costs roughly five times a single chat turn at the same model. Forecast in expected tokens per session, not requests per second.
The real economics of inference
Frontier models (GPT-class, Claude Sonnet/Opus, Gemini Pro) price for reasoning quality. Mid-tier models (GPT-mini, Claude Haiku, Gemini Flash) price for throughput. Open-weights hosted endpoints (Llama, DeepSeek, Mistral) price for floor cost. The right architectural move is rarely 'pick the cheapest model' — it's routing: cheap models for the 80% of low-complexity calls, frontier models for the 20% that need them. A well-routed product typically lands 40–70% below a single-model baseline.
RAG, embeddings, and the cost of memory
Embeddings are cheap to compute and cheap to store, but every retrieval call costs LLM input tokens. A 10K-document corpus with 2K-token chunks and top-5 retrieval injects roughly 10K input tokens per query — often more than the user's prompt. Vector databases add a small per-query fee on top. The lever is chunk size and retrieval depth, not the embedding model.
Unit economics for AI products
A defensible AI product needs gross margin north of 60% at steady-state usage. Calculate it as (price − COGS) / price, where COGS = average tokens per user-month × blended model price + vector storage + observability. If you're below 40%, the fix is almost always model routing, prompt compression, or response caching — not negotiating with the provider.