TokenAtlasPer-conversation cost
Typical support conversation: 8 turns × ~400 tokens = ~3.2K tokens. On GPT-4o-mini, that's ~$0.001 — i.e. $1 per 1,000 conversations. On GPT-4o it's $0.012 — $12 per 1K.
Where cost runs away
Long system prompts re-billed every turn. Unfiltered retrievals dumping 20K tokens into context. Customers chatting for 50+ turns. Cap turn count and cache aggressively.
Model selection by intent
Greeting + FAQ → Haiku/Flash. Account actions + tool use → GPT-4o-mini. Complex troubleshooting escalation → Sonnet or GPT-4o. Routing saves 60-80%.
Worked example with stated assumptions
Assume 8 turns/conversation, 300 input + 150 output tokens per turn, 20,000 conversations/month: that's 48M input and 24M output tokens. On Claude 3.5 Haiku (0.7x$0.80 + 0.3x$4 blended = $1.76/1M): roughly 72M total tokens x $1.76/1M ≈ $127/month, using the blended rate as a shortcut once the input/output split is fixed at this ratio.
What is not captured by turn count alone
Two conversations with the same turn count can cost very differently if one carries a 2,000-token system prompt re-billed every turn and the other doesn't. Retrieval-augmented turns that inject document context add input tokens that scale with the retrieved chunk size, not with the user's message length — model that separately from plain chat turns.
Sensitivity: what happens if turns double
If average conversation length grows from 8 to 16 turns as users get comfortable with the bot, both input and output volume roughly double, and so does the modelled cost — unless the system prompt is cached, in which case only the marginal turns' tokens grow. Running this scenario before launch shows whether a length cap or caching is worth building in from the start.
Translating per-conversation cost into per-agent-hour saved
If a human agent handles roughly 6 conversations/hour at a fully-loaded cost of $25/hour, that's about $4.17/conversation in labor cost — compare that directly against the modelled $0.001–$0.012 AI cost per conversation on this page. This per-unit framing is what justifies a deflection strategy to a finance stakeholder, rather than the raw token-cost figure on its own.
Assumptions checklist for a chatbot use-case estimate
— Conversations per month by intent category (FAQ, account action, escalation), since routing changes the blended rate — Average turns and tokens per turn within each category — System prompt length and whether it's cached — Escalation rate to a human agent, which determines the AI-cost ceiling per resolved ticket
What this page does not do
This page models per-conversation and per-category economics from volumes and token shapes you provide; it does not implement the intent router, connect to a helpdesk platform, or measure actual deflection rates. Those require the chatbot's own analytics — use this page to set a cost budget per intent tier before building the routing logic.
Frequently asked questions
- What's a fair budget per conversation?
- Support: $0.01–$0.05. Sales: $0.05–$0.20 (worth the spend if it converts).
- Should I cache?
- Always for system prompts >2K tokens. Caching is the highest-leverage cost lever in chatbots.
- How do I prevent abuse?
- Per-user token caps + rate limits + a fallback to a smaller model after N turns.