A coding agent often feels slow long before the model itself is the only bottleneck. Each turn has to reintroduce a working scene: base instructions, tool schemas, project rules, permission state, prior messages, tool outputs, and the new user request. In a long task, the cost of rebuilding that scene becomes visible.

The official OpenAI prompt caching guide gives the provider contract: cache hits require exact prefix matches; stable content belongs at the beginning; dynamic content belongs near the end; prompt_cache_key can influence routing; and usage.prompt_tokens_details.cached_tokens is the observable result. Codex source then answers a narrower question: how does the client build a repeatable model-visible request?

The working claim is simple: Codex prompt caching is request-shape discipline. prompt_cache_key helps similar requests land in a useful cache domain, but the hit still depends on the ordered request fields: instructions, tools, input history, context updates, tool output, and compaction shape.

Evidence boundary. OpenAI docs define the provider-level prompt caching contract: exact prefixes, eligibility threshold, routing, retention, prompt_cache_key, and cached_tokens. Public Codex source shows request construction, history persistence, compaction, and token usage accounting. This article does not infer private server serialization, machine routing, or KV-cache placement from client code.

This part follows six questions:

  1. Which view is actually cacheable?
  2. Where does Codex get prompt_cache_key, and why is it thread-scoped by default?
  3. How do instructions, tools, and input form a stable prefix?
  4. Why should settings, environment context, and tool output land after that prefix?
  5. Why does compaction reduce pressure while changing future cache shape?
  6. Which metrics connect cache behavior to perceived latency?

1. Cache the Model View, Not the UI Transcript

Asking whether a turn “hit the cache” is too coarse. First ask: what was sent to the provider, in what order, and through which fields? The visible UI history, the rollout persisted on disk, and the request assembled for the model are related surfaces, but they are not identical.

Layer Owner Role Cache relevance
Visible history Client projection Shows turns, tools, hooks, events, and final answers. Useful for reading; not proof of the next model input.
Durable record Rollout / thread store Supports resume, replay, fork, and audit. Seeds reconstruction, but is not the provider cache.
Model view Prompt / Responses request Contains this turn's instructions, tools, input, and controls. The provider can only reuse this ordered request view.

Codex makes this separation visible in Prompt. It carries conversation input, model-visible tools, parallel_tool_calls, base_instructions, personality, and output schema. ModelClient::build_responses_request turns that into a Responses payload: instructions, input, tools, reasoning, text, and prompt_cache_key. 

Shape-level request:
Responses request
  instructions: stable base instructions
  tools:        model-visible tool schemas
  input:        prior context + dynamic turn tail
  text:         output schema / verbosity
  prompt_cache_key: thread-scoped cache domain

2. The Cache Domain Comes From the Thread

ModelClient is session-scoped. Its comments say it holds cross-turn state such as auth, provider selection, thread id, and transport fallback state. That scope explains its default prompt cache key.

ModelClient::prompt_cache_key() returns an override when one exists; otherwise it uses the thread_id. build_responses_request places that value on every Responses request. Consecutive turns in one thread therefore share a stable cache domain unless a higher-level session type has a reason to override it.

prompt_cache_key should not be read as a manual breakpoint. It does not tell the provider where to cache. It keeps related requests near the same routing domain so repeated prefixes have a better chance to be found.

Guardian review sessions are the visible exception: session setup can call with_prompt_cache_key_override. That keeps the key tied to a request family, not to a one-off user message.

3. Stable Prefix: Instructions, Tools, Context Base

OpenAI's best practice is to put stable content first. Codex has three obvious candidates: base instructions, tool schemas, and the initial or diffed context baseline.

3.1 Instructions Stay Stable Across Ordinary Turns

build_responses_request reads prompt.base_instructions.text into instructions. The prompt caching test suite submits two ordinary turns and checks that instructions remains unchanged. When a model path needs stable apply-patch guidance because the tool itself is absent, the test also checks that the appended instruction block remains identical.

3.2 Tools Are Prefix Material

OpenAI docs say tool definitions can be cached and count toward the eligibility threshold. Codex converts prompt.tools into the Responses tools field. Parts IV and VII already showed why that list can be large: core tools, MCP tools, deferred tool search, dynamic tools, and plugin or skill supplied capabilities all feed the model-visible tool surface.

A large tool surface has a cost. But a stable tool surface is also excellent prefix material. Unnecessary per-turn tool churn hurts cacheability before the model even looks at the new user input.

3.3 Context Base: Inject Once, Diff Later

The context-management part introduced reference_context_item. It is the baseline for later context diffs. When the baseline is still valid, Codex can append an update rather than reinjecting the whole context bundle. The test named prefixes_context_and_instructions_once_and_consistently_across_requests demonstrates the effect: the second request preserves the first request's input prefix and appends the next user message.

4. Dynamic Tail: Change Later, Not Earlier

Dynamic material is inevitable. Users switch settings, change cwd, change approval policy, request a different model, trigger tools, receive tool outputs, or let hooks add context. Codex does not try to remove all change. It tries to keep change behind the already-stable prefix.

Two tests make that concrete. One applies thread settings overrides and checks that prompt_cache_key stays constant while updated permissions and environment messages are appended after the existing prefix. Another applies per-turn overrides and checks the same invariant while adding a model switch and new environment context.

Change source Codex handling Protected property
New user input Append a new user message. Prior prefix is not reformatted.
Thread settings override Append settings and environment updates. Stable context remains before the change.
Tool output Record history, then normalize with for_prompt(). Call/output invariants stay intact.
Hook additional context Write a contextual fragment into model context. Policy additions become visible input changes.
Compaction Install summary or replacement history as a new baseline. Future turns get a smaller but reconstructible prefix.

5. Compaction Relieves Pressure and Changes Shape

Prompt cache does not replace the context window. Once history grows too large, Codex still has to compact. ContextManager stores token usage, estimates counts, and bumps history_version when history is rewritten. The get_context_remaining tool exposes the remaining window by subtracting active token usage from the model context window.

In remote compaction v2, Codex clones history, reads base instructions, and trims function-call history to fit the window before building the compact request. The compact path then reuses the same shared request fields. Tests assert that the compact request carries a string prompt_cache_key and that it matches the normal Responses request.

Compaction has two performance effects: it reduces future history pressure, and it establishes a new prefix shape. The old prefix is no longer the mainline model view; the summary, retained outputs, and reinjected context become the next cache candidate.

6. Metrics: Cached Input, Non-Cached Input, First Token

The provider reports cache behavior through usage. Codex tracks input_tokens, cached_input_tokens, output_tokens, reasoning_output_tokens, and total_tokens. At turn completion, on_task_finished compares the token snapshot from turn start with current total usage to compute this turn's input, cached input, non-cached input, output, and total tokens.

The same path records tracing fields, session telemetry histograms, analytics events, time_to_first_token_ms, and total turn duration. That matters because a high cached-token count and a fast-feeling turn are related but not equivalent. Tool loops, approvals, network retries, compaction, and output length all shape user-visible latency.

Metric Meaning Do not read it as
cached_input_tokens Input tokens served from provider cache. Tokens the model did not see.
non_cached_input() Input tokens that were not cache hits. All non-cached tokens are waste.
time_to_first_token_ms Delay until the first visible model output. A value controlled only by prompt cache.
total_tokens The turn token ledger. A guarantee of semantic completeness.

7. Common Misreadings

The API fields look small, but the runtime discipline behind them is large. These are the misreadings that tend to make performance debugging noisy.

Misreading Better reading Source boundary
prompt_cache_key decides the hit. The key helps routing; exact prefix still decides eligibility. Key construction and request shape are separate.
If the chat shows it, the model sees it. The model sees for_prompt() output. History normalization owns the request input.
Fewer tools are always faster. Stable tool schemas are costly but cacheable prefix material. The tools field comes from prompt.tools.
Compaction automatically improves cache. Compaction changes the future prefix and must remain reconstructible. Remote compact reuses the key while rewriting history.
A cache hit guarantees a fast turn. Tool loops, approvals, compaction, output length, and network still matter. Codex records token usage, TTFT, and duration separately.

8. Why Recovery Comes Next

Prompt cache pushes performance analysis onto model view shape. But that shape has to be reconstructed from durable history: rollout items, context updates, tool outputs, compaction replacement, rollback markers, and fork state. If that durable record cannot rebuild the same history, the next request shape drifts and cache analysis loses its anchor.

The next part returns to persistence and recovery: how Codex writes a turn into rollout, reconstructs history from RolloutItem, handles rollback, fork, compact boundaries, and token usage. That closes the loop between what the user saw, what disk retained, and what the model sees on the next turn.

Source References