Context engineering is the new prompt engineering

From art to engineering

Prompt engineering was about finding magic words. Context engineering is about managing a scarce, priced, position-sensitive resource, much closer to memory hierarchy design than to copywriting. The papers gave us the constraints; production gave us the invoice. This post is the merge of the two.

Constraint 1: position matters

Liu et al.'s "Lost in the Middle" result is old news by now, but its production corollary isn't widely practiced: models attend best to the beginning and end of context, worst to the middle. Modern frontier models flattened the curve, but they didn't eliminate it. On our internal retrieval evals, moving the critical document from the middle to the end of a 100k-token context still buys a measurable accuracy bump.

Practical layout, top to bottom: identity and rules → tools → curated reference → conversation → the live task, restated last. If something must not be missed, it goes at an edge, usually the bottom edge.

Constraint 2: tokens are priced by position too

Prompt caching charges you a premium (~1.25×) to write a prefix once, then ~0.1× to reread it every turn after. The catch: the cache is a prefix cache. One changed byte at position N invalidates everything after N.

Design rule: sort your prompt by volatility. Anything that changes per-turn (timestamps, mutable state, retrieval results) lives after everything that doesn't. We've seen a single "current time" line at the top of a system prompt double a team's inference bill.

def assemble(system, tools, memory, transcript, task):
    return [
        block(system,  cache=True),   # never changes
        block(tools,   cache=True),   # changes on deploy
        block(memory,  cache=True),   # changes on compaction
        block(transcript),            # append-only
        block(task),                  # volatile — always last
    ]

Constraint 3: attention is a budget

Even within the window, every token competes for attention. Fill the window to 95% with tool logs and the model gets measurably dumber at the actual task, a problem known as context rot. Two mitigations do most of the work:

• Compaction. When the transcript crosses a threshold, summarize it into a structured state block (decisions, open questions, file map) and drop the raw scroll. Aggressive? The model that wrote the summary is the model that reads it.
• Memory files. Facts that must survive sessions go to disk, not context. An index loads every session; bodies load on demand. Context becomes a cache over durable state, which is exactly what it economically is.

def should_compact(ctx):
    return (
        ctx.tokens > 0.7 * ctx.window
        or ctx.tool_output_fraction > 0.5   # transcript is mostly logs
        or ctx.turns_since_compaction > 30
    )

Context is a cache over durable state. Treat it like memory hierarchy, not like a chat log.

Constraint 4: don't read it yourself

The highest-leverage trick of 2026 barely appears in papers: subagent fan-out as a context firewall. When a question requires reading fifty files, the orchestrator shouldn't read them. It spawns a searcher that burns its own disposable window and returns three sentences. Reading happens in a sandbox; only conclusions enter the main line.

Our numbers on a repo-audit workload: orchestrator context stayed under 60k tokens across a task that touched ~1.4M tokens of source. Cost fell 38% versus single-context, and accuracy rose. The orchestrator never got lost in someone else's middle.

The checklist

• Sort by volatility; breakpoint the stable prefix; never edit above the fold mid-session.
• Put must-not-miss material at the edges, preferably the end.
• Compact on shape (log fraction), not just size.
• Persist facts to memory files; context is a cache, not a home.
• Fan out the reading; keep the conclusions.
• Watch cache-hit rate in your metrics dashboard like you watch p95 latency. It's the same kind of number: silent, boring, and worth thousands.

None of this needed a new model. It needed treating the context window as what it is: the most expensive megabyte you rent all day.