og.cc.cd / guides / reduce-token-usage-prompt-caching

Reduce LLM Token Usage: Prompt Caching Basics for OpenAI, Claude, Gemini and DeepSeek

By og.cc.cd · Published · Updated · 4 min read

TL;DR: Most LLM spend is repeated input; trim what you send, then put stable content (system prompt, tools, documents) first so OpenAI, Claude, Gemini and DeepSeek prompt caching can bill it at the cheaper cache-hit rate.

Updated · All sources ↓

Most LLM spend is repeated input: the same system prompt, tools and documents sent again on every call. This guide covers the cheap wins first, then how prompt caching works on each major API and how to structure a prompt so the cache actually hits.

Measure your prompt first →All guides

Start with the cheap wins

  • Measure before you optimise. Paste your full system prompt and a typical request into the token counter. Teams are often surprised that tool definitions or a pasted style guide are the biggest block.
  • Cap output. Set a max-tokens limit and ask for the format you need (“return JSON with these three fields”). On the models we track, output is billed at roughly 3× to 8× the input rate.
  • Retrieve, don’t paste. Send the three most relevant document chunks instead of the whole manual.
  • Trim conversation history. Summarise or drop old turns in long chats — but know that rewriting history breaks caching (see below), so do it rarely and deliberately.
  • Route by difficulty. Send easy classification or extraction to a small model and reserve the large model for hard requests.
  • Batch what can wait. OpenAI and Anthropic both document 50% off for batch processing.

How prompt caching works

When a model reads your prompt it computes intermediate key-value (KV) states for every token. Prompt caching stores those states for a prefix — the unchanged tokens at the beginning of a request. When a later request starts with exactly the same prefix, the provider reuses the stored state, which is faster and billed at a discount. The output is generated fresh; caching does not change the answer.

The one rule that matters: caching matches an exact prefix. Put stable content first (tools, system prompt, reference documents) and changing content last (timestamps, user names, the new question). A single changed character early in the prompt invalidates everything after it.

Vendor by vendor

Anthropic (Claude)

  • Opt-in: add one top-level cache_control: {"type": "ephemeral"} for automatic caching, or place cache_control on specific blocks (up to 4 breakpoints).
  • Pricing: 5-minute cache writes cost 1.25× base input, 1-hour writes 2×, and reads 0.1× — except reads on Claude Fable 5.1 (0.025×) and Claude Opus 5.5 / Sonnet 5.5 (0.05×).
  • Minimum cacheable prompt: 512 tokens for the 5.x models such as Opus 5.5, Sonnet 5.5 and Haiku 5.5 (older models need more). Shorter prompts are silently not cached.
  • Lifetime: 5 minutes by default, refreshed free on every hit; 1-hour TTL available. Check cache_creation_input_tokens and cache_read_input_tokens in usage.

OpenAI

  • Enabled by default on supported models. For GPT-5.6 and later the minimum cacheable prefix is 1,024 tokens.
  • For GPT-5.6 and later, cache writes cost 1.25× uncached input and reads 0.1× (0.05× on gpt-6.1-sol).
  • Implicit mode places a breakpoint at the end of the latest eligible message; prompt_cache_options.mode: "explicit" plus prompt_cache_breakpoint lets you choose. Lifetime is at least 30 minutes after the last write or reuse.
  • Watch usage.input_tokens_details.cached_tokens and cache_write_tokens.

Google (Gemini)

  • Implicit caching is on by default for Gemini 2.5 and newer, with nothing to configure; savings are passed on automatically on a hit.
  • Minimum input is 4,096 tokens for Gemini 4 Argon and 3.8 Flash (2,048 for 2.5 Flash / Pro).
  • Explicit cache objects are not supported for Gemini 4 Argon onward or in the Interactions API. Check usage.total_cached_tokens.

DeepSeek

DeepSeek’s pricing table lists separate “cache hit” and “cache miss” input prices. For deepseek-flash at peak, a cache hit costs $0.006 per million input tokens versus $0.30 on a miss — a 50× difference — so the same stable-prefix discipline pays off here too.

Worked example

A coding assistant on Claude Sonnet 5.5 ($2 per million input) sends a 20,000-token prefix (system prompt, tools, project docs) on 100 requests within the cache lifetime.

Without caching: 100 × 20,000 × $2 / 1M            = $4.00
With caching:    1 write  × 20,000 × $2.50 / 1M    = $0.05
                 99 reads × 20,000 × $0.10 / 1M    = $0.198
                 total                             ≈ $0.25   (about 94% less on the prefix)

The variable part (the user’s new question and the answer) is billed normally in both cases. The saving only appears if requests arrive inside the lifetime and the prefix is byte-for-byte identical.

Before and after: ordering a prompt for caching

Here is a typical prompt layout that never hits the cache, and the same content reordered so that it does. Only the order changed.

# Before: the prefix changes on every request
system: "Today is 2026-10-18 14:03. User: Alice (plan: Pro).
         You are a support agent for Acme... [3,000 tokens of policy]"
tools:  [search_orders, refund]          # order varies by request
user:   "Where is my order #1234?"

# After: stable first, variable last
tools:  [refund, search_orders]          # fixed order, fixed schemas
system: "You are a support agent for Acme... [3,000 tokens of policy]"
        <- cache breakpoint here (Anthropic cache_control / OpenAI explicit)
user:   "Context: today is 2026-10-18; user Alice, plan Pro.
         Question: Where is my order #1234?"

On Anthropic, the cache prefix is built in the order tools → system → messages, so tool definitions belong at the very front. On OpenAI’s GPT-5.6 and later, an explicit breakpoint after the stable block means the changing suffix is processed at the normal rate without a cache-write charge.

Verify that it works

  1. Send the same request twice within a minute.
  2. On the second response, the cached-token field (Anthropic cache_read_input_tokens, OpenAI cached_tokens, Gemini total_cached_tokens) should be close to your prefix length.
  3. If it is zero, check the minimum length, then diff the two requests byte by byte — something early in the prompt is changing.
  4. In production, track cached tokens ÷ total input tokens per day; a sudden drop usually means a deploy changed the prefix.

Common ways to break the cache

  • Putting the current date, a request ID or the user’s name at the top of the system prompt.
  • Re-ordering or editing tool definitions between requests (both OpenAI and Anthropic list tool changes as cache-invalidating).
  • Serialising JSON with unstable key order — Anthropic specifically warns about languages that randomise map order.
  • Switching reasoning effort, thinking settings or images on and off mid-conversation.
  • Prompts just below the minimum length: on OpenAI and Anthropic it can be cheaper to add useful, stable examples and cross the threshold.

For agents, the same ideas apply to instruction files: a short, stable CLAUDE.md or AGENTS.md loaded at the start of every session costs less than one that changes every day.

Sources

Vendor features and prices were checked against these official pages on 2026-10-18. Products change; if something here no longer matches, the official page is right.

Related guides