Bounded-context orchestration model

Type: kb/types/note.md · Tags: computational-model

Closed-world LLM workflows admit a batched select/call normal form when they meet three conditions. Their non-LLM execution between batch barriers is symbolic. Their transition-relevant state is explicit. Calls released in the same batch do not affect one another before the barrier. Under these conditions, code selects a nonempty batch, runs its calls concurrently, waits for all of them, records their results, and selects again. A singleton batch represents sequential execution. No one call must receive the full workflow state.

The form represents qualifying workflows; it does not rank orchestration architectures. It exposes selection as a policy that designers can compare. Any comparison must identify what remains fixed and what outcome should improve.

Why separate the scheduler from the calls

Designers may choose an external symbolic scheduler for two reasons. First, context is scarce. Prompt capacity and soft degradation limit the evidence and complexity that one call can use, while external state can retain more than the call receives. Second, code can apply fully specified transitions without the stochastic omissions and mutations that LLM bookkeeping can introduce. Whether this difference yields a net reliability advantage remains conjectural, because code also creates specification, maintenance, synchronization, and interface costs. These reasons motivate the architecture; they do not prove that it is superior.

The model

The form has six elements:

  • Scheduler state K stores source artifacts, control position, and retained results outside any one call's context window.
  • Call specification C identifies a prompt, task, model, and any execution capabilities that can change the call's behavior.
  • Batch B = (C_1, ..., C_n) groups a finite, nonempty indexed family of call specifications selected from the same state. Its members have no transition-relevant interaction before the batch barrier.
  • select(K) returns the next batch. It returns None when no more LLM calls should run.
  • call_all(B) runs every call(C_i) concurrently. After all calls finish, it returns the aligned results R = (r_1, ..., r_n).
  • transition(K, B, R) records the completed results and advances the scheduler state.

The loop treats prompt feasibility as a relation among a call's complete prompt, task, model, and required performance threshold. The form does not reduce this relation to one measurable scalar. Token count, compositional difficulty, and task framing can each affect it, and soft degradation can bind before a hard context cap. In practice, select estimates feasibility from token counts, known prompt templates, empirical results, or earlier bounded judgments.

The minimal loop is:

while (B := select(K)) is not None:
    require independent(B, K)
    require all(feasible(C.prompt, C.task, C.model) for C in B)
    R = call_all(B)
    K = transition(K, B, R)

The conversion lemma collapses symbolic non-call execution into select when K stores the complete resumable machine state. Branches, retries, queues, phase tags, cached views, and nested loops become data in K and rules in select or transition. The conversion preserves batch membership and barrier order, so it does not turn parallel calls into serial calls.

Cases inside the form

A batch with more than one member represents parallel calls. It contains the independent calls released together at one barrier, not every call that could ever overlap. call_all waits for all members, and transition applies an explicit merge or arbitration rule to their aligned results.

LLM-assisted planning also fits. A planning call returns a result in R; transition records the plan in K; and a later select consumes it. Hierarchical decomposition repeats the same loop rather than introducing a different mechanism.

Bedi reports a ContextProvider pattern in which each provider wraps one source behind two natural-language query/update tools; a source-scoped sub-agent owns the underlying tools and source-specific operating details and may load a source skill. A provider call can be represented as a source-scoped singleton batch only when an implementation also keeps transition-relevant state explicit, schedules inter-call work symbolically, and contains effects behind the batch barrier. The retained source extracts establish the provider boundary and delegated calls, not those additional normal-form conditions. The pattern therefore illustrates a possible decomposition without establishing that every ContextProvider implementation belongs inside this form.

Boundary

The form excludes interactions that cross its barriers. One batch cannot contain calls whose streams, results, or transition-relevant side effects influence one another before completion. A tool-using call fits only when the scheduler observes its transition-relevant effects at the barrier and no hidden interaction occurs within the batch.

The form also assumes a closed world and shared barriers. K cannot exactly represent an environment that changes independently between observations, and the loop cannot preserve staggered overlaps that have no shared barrier. Streaming interaction, staggered overlap, and independently mutable environments require a richer event-driven or environment-state model.

What makes selection hard

Adaptive results create sequential dependence. A completed batch can reveal that the goal decomposes differently than expected. That result may change every later selection. Static evaluation sweeps mark the opposite limit: designers can choose every batch and merge rule in advance.

Selection and framing are coupled. Prompt cost and effectiveness depend on token volume, compositional difficulty, task framing, and the requested operation. The same documents can produce different results when one prompt merely presents them and another names a relation to test or resolve. Because information value is observer-relative, a scheduler cannot generally optimize material selection independently of task framing.

Comparing selection strategies

The form supplies no objective function. To compare two selection strategies, hold relevant semantics or evidence coverage fixed and state what should improve. Possible criteria include task success under a prompt budget, reliability under a latency bound, and auditability at a fixed success threshold. Without a criterion, the claim that a transformation moves “in the right direction” has no determinate meaning. The decomposition rules propose transformations; tests against a stated criterion determine their value.

Scope and open questions

The form applies most directly to closed-world, barriered-call workflows with explicit resumable state. The representation does not imply one global objective. Goals are underspecified, calls are stochastic, and latency, context cost, reliability, and auditability trade off. The form instead gives scoped comparisons a shared vocabulary for their invariants and criteria.

  • Which task- and model-relative feasibility predicates predict successful calls well enough to guide select?
  • When should an orchestrator retain, compress, or discard explicit state?
  • Which restrictions on branching, decomposition depth, or call interaction yield tractable strategy comparisons?
  • What event-driven extension preserves useful select/call structure for streaming calls and independently mutable environments?

Sources:

Relevant Notes: