This is not a primer on AI agent pricing models. The taxonomy of per-token, per-task, per-agent, and capability-based pricing already lives at AI agent cost models explained. This piece is the operational sibling. Your agent is in production; the bill is climbing; you have a meeting tomorrow with finance. What do you actually do this week.

Every tactic below has a quantified savings range, a config sketch, and a note on where it backfires. The ordering reflects ROI in real production deployments I have shipped or watched. Skim to the ranked-tactics section at the bottom if you want only the top three.

Why the bill climbs faster than usage

Three patterns drive AI agent cost growth out of proportion with usage. First, prompt creep: as the agent gains capabilities, the system prompt grows, and every call pays for the full prompt. Second, retry storms: a tool failure triggers a retry, which re-runs the whole agent loop. Third, context bloat: long-running agents accumulate conversation history, and the input token cost grows quadratically with the conversation length.

The cost optimization work is therefore not just "use a cheaper model". It is structural: reduce the input you pay for, reuse what you can, push asynchronous work to cheaper rails, and stop runaway loops before they finish.

Prompt caching

Prompt caching reuses the encoded representation of a stable prompt prefix. The model providers cache the prefix; subsequent calls that share the prefix pay a fraction of the input-token cost.

Anthropic prompt caching

Anthropic introduced prompt caching in August 2024. Cache hits charge 10 percent of normal input price; cache writes charge 125 percent. Latency on cached portions drops by up to 85 percent. The cache lifetime is 5 minutes by default, extensible to 1 hour (Anthropic prompt caching, 2024).

OpenAI automatic prompt caching

OpenAI rolled out automatic prompt caching in October 2024. Cached input tokens are billed at 50 percent of the normal rate. No client configuration is required for prompts over 1024 tokens (OpenAI prompt caching, 2024).

Where caching backfires

Caching only helps when the prefix is stable. Agents with personalised system prompts per user, or with frequent prompt edits, get little to no caching benefit. Restructure the prompt so the static portion (instructions, tool definitions, examples) lives at the top and the dynamic portion (user-specific data) lives at the bottom.

Model routing

Most production agent traffic does not need a frontier model. The routing pattern: a fast classifier decides task difficulty; easy tasks go to a small model; the model output is verified; if verification fails, escalate to the large model.

Two-tier routing

Tier 1 is a small model (Claude Haiku 4.5, GPT-4o mini, Gemini Flash) handling the bulk of traffic. Tier 2 is a large model (Claude Opus 4.7, GPT-4, Gemini 2.5 Pro) handling escalations. Typical split: 70-85 percent of traffic stays at Tier 1; cost drops 50-70 percent versus all-Opus or all-GPT-4.

Eval-gated downgrade

Never downgrade based on intuition. Build a 50-200 example eval set per agent capability. Run both models. Downgrade only if the smaller model passes the accuracy threshold. The eval set is the contract.

Where routing backfires

If verification is brittle, you end up paying for the small model and the large model on the same task. Verification must be cheap and reliable. A regex on output schema is cheap; a judge-model verification is not.

Batch APIs and async patterns

For workloads that do not need a synchronous response, the batch APIs cut cost in half.

OpenAI Batch API

50 percent of synchronous pricing, 24-hour completion window (OpenAI Batch API documentation).

Anthropic Message Batches

50 percent discount on input and output tokens, 24-hour completion window (Anthropic Message Batches API, 2024).

Workloads to push to batch

Overnight reports, bulk classification, periodic data syncs, document summarisation for tomorrow's review. Anything where the user is not waiting on the result.

Prompt compression

Two flavours: structural and learned. Structural compression rewrites the prompt: shorter instructions, fewer examples, removed redundancy. Learned compression uses techniques like LLMLingua to remove tokens that contribute little to the model's response (LLMLingua, Jiang et al., 2023). Structural compression is cheap and worth doing routinely. Learned compression is buyer-dependent and benefits drop when prompt caching is already enabled.

Budget guardrails

Caps are cheap insurance. Three layers.

Per-run cap

The single most important guard. Each run carries a maximum spend. Beyond the cap, the agent halts and escalates. Catches runaway loops and cost-bomb prompt injection in the same control.

Per-agent-per-day cap

An agent that normally costs five dollars per day and suddenly costs five hundred is a signal. The daily cap is a circuit breaker.

Per-tenant-per-month cap

If you charge per tenant, you must cap per tenant. Graceful degradation when a tenant hits cap: pause new runs, finish in-flight, notify the customer.

For the security view on budget caps see AI agent security best practices.

Per-tenant attribution

You cannot decide pricing without unit economics. Every tool call and every LLM call carries a tenant tag. Cost aggregates roll up by tenant per day, per month, lifetime. The output drives pricing decisions: which tenants are profitable, which are subsidised, which need a different plan.

For the strategic pricing view see AI agent cost models explained.

Tactics ranked by savings

TacticTypical savingsEngineering effort
Prompt caching30-70 percent of input costLow (prompt restructure)
Two-tier routing40-60 percent of total costMedium (eval set + classifier)
Batch API for async50 percent on async portionLow (API swap)
Structural prompt compression10-30 percent of input costLow
Eval-gated model downgrade20-50 percentMedium
Budget guardrailsVariable (catches outliers)Low
Learned prompt compression5-15 percent when caching already onMedium

Frequently asked questions

How do I reduce AI agent costs in production?

Turn on prompt caching, route easy tasks to a small model with an eval-gated fallback, add a per-tenant budget cap. Prompt caching alone delivers up to 90 percent input-token savings on cached portions.

What is prompt caching and how much does it save?

Prompt caching reuses the encoded representation of a stable prompt prefix. Anthropic prompt caching cuts cached-portion input cost by up to 90 percent. OpenAI automatic prompt caching applies a 50 percent discount on cached input tokens.

When should I downgrade to a smaller model?

When a held-out eval set shows the smaller model meeting your accuracy threshold. Never downgrade on intuition. The eval set is the contract.

How do I set a budget cap on an AI agent?

Per-run, per-agent-per-day, and per-tenant-per-month. Per-run is the most important; it stops runaway loops and cost-bomb injection.

Is batch processing cheaper for AI agents?

Yes. Both OpenAI and Anthropic offer batch APIs at roughly 50 percent of synchronous pricing.

Three things to ship this week

Sources