description: Agent context is bounded by silent reliability degradation across volume, complexity, and relevance/interference, not just by provider token limits type: kb/types/note.md traits: [has-external-sources, title-as-claim] tags: [learning-theory, foundations, deploy-time-learning]

Agent context is constrained by soft degradation, not hard token limits

Agent context windows have two bounds: a hard token limit and a soft degradation surface. The hard limit is the maximum tokens the model accepts — exceed it and the API rejects the request. The soft bound is where performance silently degrades: missed instructions, shallow reasoning, ignored context — while output remains well-formed.

The soft bound is the binding constraint — performance degrades well before the hard limit is reached. What constrains work is not running out of tokens but the quality of what those tokens do, driven by at least three dimensions: volume, complexity, and relevance/interference. Other factors — information arrangement and prompt framing — also shift the degradation surface, often by changing one of these dimensions indirectly.

Dimensions of the soft bound

Volume

More tokens dilute attention. The "lost in the middle" finding (Liu et al., 2023) established primacy and recency bias — models overweight information near the beginning and end of the context, underweighting the middle. Because agent prompts face the same flat-sequence selection problem, this positional bias applies whenever the model must recover the right items from a long unscoped context. Anthropic (the AI lab) calls this context rot (2025). Paulsen's Maximum Effective Context Window (MECW) work confirms that usable context can be far below advertised windows and is task-dependent (Paulsen, 2025).

Relevance/interference

Not all tokens are equal. Irrelevant context is not merely extra volume; it can actively interfere with task execution. GSM-DC, a math-reasoning benchmark with synthetic distractors, shows power-law error scaling with distractor count (Yang et al., 2025). The interaction with reasoning depth is the key signal: distractors hurt more as the task requires more dependent steps, and they degrade both reasoning path selection and arithmetic execution.

The same pattern appears at the agent-workflow level. Chung et al. find that injecting irrelevant task sequences into web-agent benchmarks collapses success rates from 40-50% to under 10% (Chung et al., 2025). The failures are not just slower retrieval from a larger context; agents loop, lose objectives, and treat stale history as live problem state. Bolt-on retrieval (iRAG) provides only modest improvement in that benchmark, which is weak but useful evidence that irrelevant context often needs to be excluded or scoped away rather than compensated for after loading.

This is why the mitigation is architectural. Summarization can shrink irrelevant material, but it does not by itself decide whether the material belongs in the active problem frame. Selective loading, scoped state, and sub-agent boundaries attack relevance/interference directly by preventing non-task state from competing with the task.

Complexity

Some forms of context complexity add interpretation overhead. Every layer of indirection costs context and interpretation overhead, and deeper compositional structure may impose similar costs. ConvexBench, a benchmark on compositional symbolic reasoning, shows complexity-driven collapse at low token counts: F1 dropped from 1.0 at depth 2 to ~0.2 at depth 100, even though total tokens (5,331 at depth 100) were far below context limits (Liu et al., 2026). The shared mechanism is that both agent operations and symbolic reasoning fail when the model must carry many intermediate dependencies without scoped subproblems or externalized state. Compositional depth, not volume, was the bottleneck.

Open questions

Volume, complexity, and relevance/interference are distinguishable but not fully separable — reducing volume often reduces complexity and interference as side effects.

The main unresolved question is interaction, not existence. GSM-DC cleanly shows that distractor count and reasoning depth interact in synthetic math problems; web-agent benchmarks show an agent-level analogue under long multi-session histories. We do not yet know how stable the interaction surface is across natural-language tasks, partially relevant material, or different model families.

Working hypothesis: soft degradation is workspace saturation and displacement

Our best current mechanical hypothesis is that much of the soft bound appears when a task's flexible computation must pass through a limited internal workspace. The J-space experiments identify a small, selectively engaged, verbalizable subspace that broadcasts intermediate representations to downstream computations. Its capacity is measured in tens of active directions rather than in input tokens, and what remains active depends on task demands and conceptual coherence.

On this account, the three observed dimensions are different ways of taxing the same workspace:

  • Volume increases the number of candidate distinctions that may need to remain available. In the paper's list experiments, only around six previously seen unrelated items remain visible at a time, while older items drop out as the list grows. Related items fare better because the workspace can represent their shared category rather than retain each item independently.
  • Relevance/interference concerns competition for admission and persistence. A new semantic category rapidly evicts the previous category, and maintaining an unrelated concept interferes with solving a multi-step arithmetic problem. Irrelevant context can therefore hurt without consuming the whole token window: it can displace or redirect the representations that flexible task execution needs.
  • Complexity increases the number and dependency structure of intermediate representations that must be broadcast. J-space ablation selectively damages multi-hop and other flexible tasks while leaving many automatic tasks intact. Explicit chain-of-thought makes GSM8K more robust to the ablation, consistent with intermediate state being externalized into tokens instead of carried wholly in the internal workspace.

The same mechanism would explain why the soft bound is hard to observe. Under J-space ablation, ordinary prediction is often preserved and disrupted continuations remain fluent; they instead fall back toward generic priors after losing an abstract, context-specific representation. A model can therefore cross its effective workspace bound without producing a visible overflow signal.

This is a hypothesis, not yet a general explanation of long-context degradation. The paper does not vary long-context workload against J-space occupancy, its lens captures only a partial and single-token-friendly view of internal representation, some automatic computation bypasses J-space, and the mechanism selecting what enters the workspace remains unknown. The hypothesis predicts that matched increases in volume, interference, or compositional depth should degrade performance insofar as they increase competition, displacement, or dependency load in the operative workspace; a degradation regime with no corresponding workspace effect would count against it.

The soft bound is invisible

The hard limit is visible — exceed it and the API returns an error. The soft bound is invisible at every level.

To the practitioner. The model doesn't signal when it crosses the soft bound. Output remains well-formed; problems surface downstream. A CPU signals overflow. A human says "I'm confused." An LLM produces fluent output whether it reflects the supplied context or leaves large portions unused.

To the benchmarker. The soft bound is not a single number. It shifts with task type, compositional depth, relevance mix, information arrangement, and prompt framing. Model updates shift the degradation surface without notice.

To the market. Providers advertise hard token limits because those are clean, comparable numbers. They don't publish soft degradation surfaces — those are task-dependent and hard to characterize. The number on the box describes the bound that rarely binds; the bound that actually constrains work has no number.

Consequences

Don't trust the number on the box. Usable context depends on what you're doing, how you arrange it, and which model version you're running.

Silent degradation makes heuristic design rational. Front-loading critical content, decomposing complexity, isolating scopes, compressing aggressively, and excluding irrelevant state are the rational strategy, not a placeholder until better measurement arrives. This is how surveyed traditions facing soft bounds have operated.

Programmatic constructability is the genuine advantage. You can programmatically choose every token that enters the context. This creates a distinctive tension: high control over inputs, low observability of effective processing. The engineering opportunity is real, but it must be exercised against a bound you cannot directly observe. Default-loading session history is the most common way this advantage goes unexercised — session history should not be the default next context. The heaviest-fork feasibility note extends these consequences to work split across sub-agents.

Relevant Notes: