After this article: you should be able to explain the path from stored information to the model’s current view, then design selection, budgets, checkpoints, and debugging traces.
Evidence boundary: this article relies on Anthropic’s official context-engineering and contextual-retrieval material and OpenAI’s official conversation-state and prompt guidance. Exact window, caching, and truncation rules depend on the current target model.
1. Understand context through the workbench story
1.1 Start with a wrong decision caused by old material
Imagine that yesterday’s payment-test log says the refund API is missing a field. The field was added this morning, but an Agent receives the old log and a long chat history instead of the current file and latest test output. It “fixes” a problem that no longer exists.
The model did not necessarily reason badly. The system prepared the wrong material. Context is the information actually available to the model for the current decision: instructions, the task, selected files, tool results, and relevant history. Context engineering is the work of selecting, ordering, updating, and compressing that material so the next step has enough evidence without being drowned in noise.
1.2 Think of a library, a candidate shelf, and a workbench
A library may hold every manual and old case file. A search first brings possible matches to a shelf. A worker then places only the few useful pages on the workbench. Agent systems have the same three levels:
- Memory or durable storage keeps information that may matter later.
- Retrieval finds possible material for the current task.
- Context is the final packet placed in the model request.
These words are often mixed together, but they describe different jobs. Saving a fact does not guarantee the next model call can see it. Finding a document does not guarantee it is current or useful. Context engineering owns that final choice.
A typical context packet may include the stable rules, current goal, completion criteria, latest observations, relevant code, and a short record of earlier decisions. It should not automatically include every message and full log.
This also clarifies its relationship to the previous article: the prompt is the relatively stable behavioral guidance inside the packet; context is the whole packet visible to this model call. They overlap but solve different problems. One makes the requirements clear; the other makes those requirements and current evidence visible at the right time.
2. Put the right material into the next decision
2.1 From storage to visibility: three separate choices
Separate three sets. A durable store retains facts that may matter later. A candidate set comes from retrieval, recent messages, and runtime state. The model view is what the request actually serializes. A successful write proves the first set contains a fact; it says nothing about the third.
| Material | Keep durably? | Carry every turn? | Main risk |
|---|---|---|---|
| Stable system rules | Yes | Usually | Diverging copies |
| User goal and done criteria | Yes | Usually | Constraints lost during compaction |
| Full tool logs | Yes, for audit | No; select relevant spans | Noise displaces evidence |
| Current file content | Repository is durable | Select by task | Stale snapshots |
| Interim plan | Checkpoint when useful | Current phase only | Old plan resists new evidence |
| Secrets and sensitive data | Minimize by policy | Default no | Disclosure and authority expansion |
2.2 How one context packet is assembled
Anthropic frames context engineering as maintaining the optimal set of tokens for every step of an agent. “Optimal” does not mean highest vector similarity. It means useful for the decision now: relevant, sufficiently fresh, trustworthy, aligned with the goal, and worth the budget.
Keep following the payment-test task. During verification, candidates include the goal, a stale error log, the current diff, the latest test output, and full chat history. The assembler removes the stale log, prioritizes the diff and latest test, accounts for fixed instructions and tool schemas, reserves output space, and only then spends the remaining budget on evidence.
New results return through the same path. After the harness runs the tests, it creates a fresh observation. The runtime stores the full log and adds the exit code, critical errors, and log reference to the candidates. On the next assembly, that item is selected because it is new and directly affects the current decision, while the old error log is removed. The model receives a workbench updated by evidence, not an unchanged replay of old chat.
def assemble(candidates, phase, window, fixed_input, output_reserve):
usable = filter_by_scope_freshness_permission(candidates, phase)
ranked = rank_for_current_decision(usable)
budget = window - tokens(fixed_input) - output_reserve
return pack_without_splitting_evidence(ranked, budget)
verify_candidates = [
"goal", "old-error-log", "current-diff",
"latest-test-output", "full-chat-history"
]
# selected: goal + current-diff + latest-test-output2.2.1 Start with the next decision
Investigation, editing, and verification need different working sets for the same task. Investigation needs entry points and error logs. Editing needs the target symbol, callers, and constraints. Verification needs the diff, test commands, and failure output. A broad task query often returns material that is globally related and locally useless.
2.2.2 Filter by provenance and freshness
Similarity does not establish authority. A formal schema outranks an old discussion; a test run from this turn is fresher than a guess from three turns ago; a workspace file is closer to current truth than model memory. Each item should carry source, time or version, scope, and sensitivity so the assembler can apply deterministic gates.
- Produce the fact: the harness reads the current
payment.gosignature together with the workspace version. - Register a candidate: the runtime puts the content or reference, provenance, version, and scope into the candidate set.
- Pass the gates: the assembler checks task relevance, freshness, and whether this model call may see the item.
- Build the view: only a selected, budgeted item enters the model view; a later file change must invalidate it and trigger a fresh read.
{
"item": "payment.go: validateRefund(...) signature",
"source": "workspace_file",
"version": "git:7ad12f + dirty",
"observed_at": "turn:18",
"scope": ["refund-fix"],
"trust": "local-authoritative",
"ttl": "until-file-change"
}until-file-change is not a promise the text can enforce. The runtime must watch or compare workspace versions and invalidate the old item after a file changes; otherwise the TTL is only metadata.
2.3 When everything does not fit, reserve space by purpose
“When tokens exceed N, drop the front” damages constraints at random. First account for fixed input such as instructions and tool schemas, then reserve output space, and only then assign the remaining budget to stable contracts, current goals, recent observations, retrieved evidence, and history summaries. Budgets can adapt, but precedence must be explicit.
| Zone | Budget principle | Under pressure |
|---|---|---|
| Behavior contract | Small, stable, high priority | Deduplicate; do not casually summarize |
| Goal and done criteria | Visible every turn | Compress structurally and validate fields |
| Current observations | Closer to the decision ranks higher | Keep error codes, key lines, provenance |
| Process history | Only state that still affects a decision | Summarize, mark invalid with a tombstone, or remove from model view |
| Candidate documents | Pack for the current phase | Defer, page, or retrieve again |
| Output space | Reserve before calling | Do not fill input and hope completion fits |
3. Advanced: compact, recover, and debug long tasks
3.1 Long tasks need compaction without losing the task
Long tasks eventually need compaction. A good compact state preserves the goal, constraints, confirmed facts, changes made, unresolved questions, next action, and evidence pointers. It can discard repeated explanations, full low-value logs, and hypotheses superseded by facts. Because summarization is lossy, the compact result must remain traceable to original records.
3.1.1 Protect state with a checkpoint schema
The fields below are a shape-level schema, not a product contract:
- Goal: current task and done criteria.
- Constraints: product, safety, and user boundaries that cannot disappear.
- Confirmed: verified facts with provenance.
- Changed: files, external objects, and recorded effects.
- Open: unresolved questions and failed attempts with reasons.
- Next: the next action and why.
Free-form summaries are readable; structured checkpoints are recoverable and testable. Use both: structure protects invariants, narrative preserves causal explanation.
{
"goal": "Fix refund_should_reject_expired_order",
"constraints": ["Keep public APIs stable", "Run payment tests"],
"confirmed": [{"fact": "expired-order guard is missing", "ref": "workspace:payment.go"}],
"changed": ["payment.go"],
"open": ["payment tests have not run"],
"next": "run payment tests",
"evidence_refs": ["trace:test-run-4", "workspace:payment.go"]
}next. Removing constraints, open work, or evidence references respectively loses boundaries, creates false completion, or breaks traceability.3.2 Retrieval success is not context success
Memory persists. Retrieval finds candidates. Context is the visible set for this call. A fact written to memory still needs retrieval, filtering, and budget to enter context. A tool result may enter the current context without becoming long-term memory.
Anthropic’s contextual retrieval highlights how isolated chunks lose document context and can be enriched before indexing. The runtime still has work after a hit: check version, permission, duplication, and relevance to the current phase. Retrieval recall is not model-view quality.
3.3 When the Agent uses the wrong fact, trace the selection
When an agent uses the wrong fact, we need to ask whether the correct fact was among candidates, where it was filtered out, why the wrong one ranked higher, which version entered the request, and whether compaction changed meaning. Each call therefore needs a context manifest that can omit sensitive bodies while retaining decisions.
- Item id, provenance, version, and selection reason.
- Deterministic filter reasons, entry or exit stage, and ranking score—not merely the final list.
- Token use by zone and every truncation event.
- Evidence pointers behind summaries and compact states.
- Redaction, authority, and retention policy for sensitive material.
4. Review context with five questions
| Review question | Failure prevented | Primary mechanism |
|---|---|---|
| What must the model know now? | Missing decisive evidence | Define the working set for the next decision |
| Where did each fact come from, and is it fresh? | Stale or false authority | Provenance, version, TTL |
| Why is it worth window space? | Noise and lost-in-the-middle effects | Zone budgets, rank, deduplication |
| Which invariants survive pressure? | Goal drift after compaction | Structured checkpoints |
| How will the next run reacquire it? | Treating a session as memory | Durable store plus replayable selection |
The outcome of context engineering is not “more tokens.” It is the right working set at every decision point. Next we move to what the model cannot see but what determines whether an action can occur: the runtime harness.
Official sources
- Anthropic: Effective context engineering for AI agents
- Anthropic: Introducing Contextual Retrieval
- OpenAI: Conversation state
- OpenAI: Prompt engineering
- OpenAI: Harness engineering