LLM Cost Dashboards
LLM API costs can surprise teams that do not instrument early. A feature that costs $0.002 per call at 100 calls/day costs $73/year - fine. The same feature at 100,000 calls/day costs $73,000/year. Understanding cost structure, building dashboards, and knowing which optimisations have the highest ROI prevents billing surprises.
Understanding LLM Cost Structure
Most API providers charge per token, with separate input and output rates. Output tokens are typically 3–5× more expensive per token than input tokens. Cached input tokens (where supported) are 10× cheaper than uncached.
| Model (mid-2026) | Input $/MTok | Output $/MTok | Cache hit $/MTok |
|---|---|---|---|
| Claude Sonnet 4.6 | $3 | $15 | $0.30 |
| Claude Haiku 4.5 | $0.80 | $4 | $0.08 |
| GPT-4o | $2.50 | $10 | $1.25 |
| GPT-4o mini | $0.15 | $0.60 | $0.075 |
Prices indicative; always check current provider pricing pages as they change frequently.
Key Metrics to Track
- Cost per request - average and P95; broken down by feature
- Cost per user - monthly spend by user cohort; surface high-cost users
- Input vs output token ratio - if output tokens are surprisingly high, prompts may be generating verbose responses
- Cache hit rate - should be high for features with static system prompts
- Cost trend - daily/weekly; alert on sudden spikes
Building a Cost Dashboard
Minimum viable dashboard from your LLM logs:
Daily total cost by model
SQL: SELECT date, model, SUM(cost_usd) FROM llm_logs GROUP BY date, model
Cost by feature (requires feature tag in logs)
SQL: SELECT feature, SUM(cost_usd), AVG(cost_usd) FROM llm_logs GROUP BY feature
Cache hit rate (Anthropic)
SQL: SELECT AVG(cache_read_tokens::float / input_tokens) FROM llm_logs WHERE model LIKE 'claude%'
Highest-ROI Cost Reductions
1. Model right-sizing (5–20× saving)
Use a cheaper small model (Haiku, GPT-4o mini) for simple tasks that do not need frontier capability. Classification, summarisation, and routing tasks often work as well on small models at 10–20× lower cost.
2. Prompt caching (3–10× saving on cached tokens)
If your system prompt is >1K tokens and is repeated across many requests, enabling prompt caching reduces cost to 10% of normal input pricing for the cached prefix. High ROI if cache hit rate is >50%.
3. Output length control (2–5× saving)
Set max_tokens to a tight limit for your use case. Prompt the model to be concise: "Respond in at most 3 sentences." Output tokens are the most expensive part of most workloads.
4. Prompt compression (10–30% saving)
Compress verbose system prompts. Remove redundant instructions, merge similar rules, and trim examples to the minimal representative set. Every 1K token reduction saves ~$0.003 per request at Sonnet pricing - significant at scale.
Budget Alerts
Set budget alerts at 50%, 80%, and 100% of your monthly budget. At 100%, consider automatic throttling rather than hard cutoffs - graceful degradation (switch to a cheaper model) is better than a 503 error. Cloud providers (AWS Bedrock, OpenAI, Anthropic) all support usage-based alerts.