Tool loop

Type: kb/types/tag-readme.md · Tags: computational-model, context-engineering, tool-loop

Many LLM applications share a common operational core: construct a task frame, give the model tools, and loop until it stops.

state = initial_task_frame()

while not done(state):
    turn = llm_call(state, tools=tools)
    if turn.type == "tool_request":
        result = execute_tool(turn.request)
        state = absorb(state, turn.request, result)
    else:
        state = absorb(state, turn.output)

Frameworks own this loop because the mechanics are repetitive protocol work — parsing tool requests, dispatching to handlers, serializing results, feeding them back, handling streaming and retries. Abstracting that away is good engineering, just as abstracting HTTP parsing is. In the bounded-context orchestration model this loop is the singleton-batch case with select frozen to one policy — append the result, re-ask with the same tools — owned by the framework rather than the application. Keeping that loop optional is the design stance these notes argue for.

Many useful interventions can stay hidden inside this loop without changing its structure: logging, approvals, budget checks, checkpoints, deterministic transforms on tool results. A stateful singleton runtime behind the tool boundary can go further, holding recursion state and branch records. The recovery is genuine — but the question is not whether the loop can absorb bookkeeping. It is who gets to decide what the next step can do.

Pressure and forcing cases

Three cases expose where progression authority lives. The first two can make a single framework-owned loop insufficient; the third distinguishes a scheduler's role from its callable interface:

Resolution

The first case calls for sub-agents — fresh tool loops with their own prompt, capability surface, and stop condition. The second calls for something more: symbolic composition of agents — code-controlled iteration, filtering, and aggregation over multiple agent invocations. The third is placement-neutral: an application loop, lower-level runtime, or explicitly named workflow tool can own the codified transitions, but the tool protocol does not make that responsibility ordinary capability implementation. Sub-agents are the atomic unit; symbolic orchestration is what the application does with them.

The framework's job is therefore to keep the tool loop optional: run the frozen loop for the common case, but expose the bounded call beneath it so application code can spawn child loops and compose them. The mechanism is small — the practical scheduler is the host language: demote the loop to a returning, per-call-parameterized agent() call (plus one tool-execution hook), and ordinary host-language control flow becomes select while its variables hold K. ("Expose the loop" was the earlier name for this same move.) A further question, once orchestrators are written this way, is what to retain across runs: run-state stays ephemeral while recurring strategies are the promotion target.

Downstream consequences

  • RLM has the model write ephemeral orchestrators over sub-agents — boundary case: the model writes an ephemeral symbolic orchestrator that composes agents via recursive_llm() — the loop is exposed to the model rather than the programmer, but discarded after each run
  • Claude Code dynamic workflows — shipped instance: a harness exposing a returning agent() plus host-language composition beneath its frozen loop — RLM's authorship model with persistence added, but in a sandboxed guest language, with capability surfaces, the dispatch interior, and the inter-workflow seams still framework-owned

Relevant Notes:

Other tagged notes