LLM-mediated schedulers are a degraded variant of the clean model

Type: note · Status: seedling · Tags: computational-model

The symbolic scheduling model assumes the scheduler is a program with unbounded exact state. In practice, many current systems (Claude Code, Codex, chat-based agent loops) carry orchestration state partly in an LLM conversation. The LLM serves as both scheduler and executor — it decides what to do next based on its accumulated conversation history.

This makes the scheduler effectively bounded: it suffers the same attention dilution and compositional overhead as the sub-agent calls it is trying to orchestrate. The clean separation between unbounded scheduler and bounded LLM calls collapses.

The framework-design consequence of this diagnosis is developed in tool loop: if conversation-mediated scheduling is the degraded case, strong frameworks should expose control surfaces that let the application layer move the loop out of chat.

Three recovery strategies

Three responses restore the separation to increasing degrees:

  1. Compaction. Keep summaries and conclusions rather than raw results in the conversation, applying distillation to the scheduler's own state. This reduces degradation but does not eliminate it.

  2. Externalisation. Write intermediate state to files and re-read selectively. This moves scheduler state out of the conversation and into exact symbolic state outside the LLM context — partially recovering the clean model.

    The Ralph Loop is a concrete pattern combining externalisation with extreme compaction: a hook intercepts the model's exit attempt and reinjects the original prompt in a clean context window, while the filesystem bridges iterations. Each loop cycle starts with zero accumulated context but reads state from the previous iteration — externalisation provides continuity, compaction (taken to its logical extreme of full context reset) prevents degradation.

  3. Factoring into code. Encode the bookkeeping and recursion as a program that runs outside the LLM conversation entirely. This fully recovers the clean model. The LLM is called only for judgment steps; the scheduler is code.

Each recovery moves the system closer to the clean model — bookkeeping, recursion, and exact state management in the symbolic layer; bounded LLM calls reserved for the semantic judgments they are uniquely needed for — and the architectural direction is toward the third option.

Relevant Notes: