Scheduler-LLM separation exploits an error-correction asymmetry

Type: types/note.md · Tags: computational-model, llm-reliability, learning-theory

Bookkeeping—maintaining task records, dependencies, counters, result identities, and completion state—determines what a workflow does next. Mistakes in these records can misdirect otherwise competent work. Code complements LLM instructions here by enforcing defined rules without asking a model to interpret them on every use. This is the scheduler–LLM separation: code maintains workflow state and enforces defined transitions, while models supply judgments and choose the work.

The conjecture is that moving this work into code can reduce the total cost of reliable execution. The mechanism is an error-correction asymmetry: code can enforce defined state transitions directly, while semantic judgments without sufficient executable checks still require fallible verification. The gain depends on whether avoided model checking and repair outweigh implementation and maintenance costs.

Bookkeeping determines what happens next

Consider an instruction to obtain the required reviews and publish a document only after acceptance. Three questions are involved:

  • Does the document answer its question well enough? That requires judgment.
  • Have all required acceptances been recorded for these exact candidate bytes? That can be checked against explicit records.
  • Are these the right reviews to require? That is a design decision.

Once the publication rule is settled, code can answer the second question by matching each review to a candidate digest. If publication must pass through this check, the system can reject a missing acceptance or one that belongs to an older candidate. Code thereby enforces the chosen rule; it does not settle whether the document is good enough or the required reviews are the right ones.

A stale acceptance matters because it can let the workflow advance without the review it was meant to require. Likewise, an incorrect completion record can cause dependent work to start prematurely or unfinished work to be skipped. Bookkeeping has leverage beyond the cost of each operation: its records are premises for later actions.

Enforcement avoids one layer of error correction

Because later actions depend on these records, their reliability matters. Clearer instructions can help a model maintain them, but do not guarantee that it will apply the rule: specification gaps and interpreter failures need different remedies. An instruction leaves each application dependent on model compliance.

Code changes how the rule is applied. Tests and runtime checks can compare identities, counts, membership, and allowed state changes directly. A tested implementation can be reused, and an invariant can be checked on every transition without an LLM call. Instead of asking a model to maintain the records and then checking whether it followed the rule, the system enforces the rule on the update itself. The companion note develops this distinction between model-mediated and symbolic operations.

This removes one checking burden, not the need for judgment. The publication gate checks the acceptance record; the reviewer still judges whether the candidate deserves acceptance. Further model reviews can help, but error correction requires discriminating checks and sufficiently decorrelated errors. Repeated agreement is not enough when reviewers share a systematic mistake.

With bookkeeping enforced mechanically, reviewers can concentrate on judging whether the result meets the task's requirements.

The model can choose the plan and revise the machinery

Enforcing bookkeeping rules does not require code to choose the work. A model can decide which tasks to create while code records them, enforces dependency rules, and tracks their results. The model can also write the code. In the RLM architecture, model-authored orchestrators run in a REPL: the model chooses the decomposition, but variables and the runtime stack carry its execution state. This illustrates the division between model-chosen policy and runtime enforcement.

The system can also improve its symbolic machinery. Running the system can reveal better state representations, missing invariants, or rules that should not have been fixed. Codification—committing a rule to symbolic execution—changes who applies it; relaxing reopens a commitment when a more general component is warranted. The system's policy can continue to change after deployment.

Scope

The argument applies to explicit state and sufficiently specified transitions. The implementation still needs testing, its inputs need to stay current, and the chosen rule needs to serve the system's objective. Synchronization, interfaces, and maintenance contribute to the total cost.

LLM bookkeeping can also be reliable. MAKER reports more than one million LLM-generated Towers of Hanoi steps with zero observed errors, using maximal decomposition, first-to-ahead-by-k error correction, and red-flagging. Its 20-disk run generated at least three responses per step. The result demonstrates reliable execution through correction in that task.

Reliable execution alone does not settle which approach costs less. The comparison must also allow for improvements to the model-mediated workflow: explicit frames and state protocols can improve a model-mediated workflow before its transitions move into code. On the code side, repeated, stable bookkeeping offers more opportunity to spread implementation costs across uses than a one-off, changing workflow.

To locate where moving bookkeeping into code pays off, a useful test would hold policy and semantic work fixed and compare total cost at a required end-to-end reliability level.

Relevant Notes: