Hosted per-token rates
Llama 3.3 70B at Together/Fireworks: ~$0.88/1M blended. Groq: ~$0.59/$0.79 with 5× the throughput. DeepInfra often undercuts both for batch workloads.
Self-hosting cost model
A single H100 sustains ~2K tokens/sec on 70B INT8. At $2/hr on-demand, that's ~$0.28/1M tokens — cheaper than hosted only above ~60% utilization.
Break-even with hosted APIs
Self-hosting wins at >5M tokens/day sustained. Below that, hosted is cheaper after factoring ops, autoscaling gaps, and engineer time.
Latency-priced providers
Groq and Cerebras charge similar $/token but deliver 5–10× throughput on Llama. Use when latency drives revenue (voice, real-time agents).
When Llama beats closed models
Privacy-sensitive workloads, regulated environments, fine-tune-heavy use cases, or sustained high-volume routes where price per token dominates.

TokenAtlas