ContextPilot boundary

Type: agentic-system-analyses/types/agentic-system-boundary.md

Boundary and evidence

This run covers ContextPilot as a context-engineering system for model-driven long-context document question answering, including its shipped inference driver, context and memory tools, prompts, evaluation callers, and reinforcement-learning improvement plane. It is not limited to the memory API. The target qualifies because model calls select operations on the working context and retained findings, and subsequent calls consume the resulting state. The repository also ships the training loop that improves those decisions (SRC-1, infer/src/contextpilot.py, infer/src/contextpilot_fsm.py, train/verl/experimental/agent_loop/statelm_agent_loop.py, train/sh/run_qwen3-8b_longbenchv2.sh).

The intended use is to characterize these source-native responsibilities and their boundaries, not to establish benchmark superiority or production reliability. The target class is memory/knowledge/context-engineering system; its included agent loops and training infrastructure are responsibilities of this system, not evidence of a general-purpose enclosing application runtime.

The boundary is whole-system: it includes the material shipped paths that ingest documents, construct searchable chunks, admit and update memories and notes, edit context, render saved content into later model inputs, terminate with an answer, save trajectories and predictions, score them, and train and checkpoint the policy. Inference's ordinary and FSM implementations are both included; the supplied task launcher selects the FSM variant. The training driver and evaluation harness are included callers, not excluded hosts (SRC-1, infer/scripts/eval_task.sh, infer/src/hf_test_runner.py, train/verl/tools/statelm_tools.py, train/verl/trainer/ppo/ray_trainer.py).

Minimum boundary evidence:

You are an AI assistant specialized in long-context document QA. Use the currently available tools to find evidence in the attached document, save useful findings, keep the context compact, and submit the final answer through finish. --- infer/configs/fsm_plan_bm25_mc_prompt.txt

This is shipped doctrine, not an instruction to this analysis worker (SRC-1). The default inference caller selects the FSM and creates a new agent for each sample, runs it, and writes answer and trajectory artifacts (SRC-1, infer/scripts/eval_task.sh, infer/src/hf_test_runner.py). The FSM's declared gating is backed by its state-driven loop:

FSM-aware run loop. At each turn only the tools allowed by the current FSM state are presented to the model. tool_choice is set to "required" in states where the model MUST call a tool. --- infer/src/contextpilot_fsm.py

Saved memory is consumed within subsequent prompt construction; its existence must not be equated with proven cross-session persistence (SRC-1, infer/src/contextpilot.py):

external_context_summary = external_memory_summary + external_note_summary --- infer/src/contextpilot.py

The training side includes memory-store tools and an explicit agent state loop, with branch trajectories subsequently consumed by the reward worker and advantage estimator. Policy and dataloader checkpoint save/load paths are included, separately from the task's memory objects (SRC-1, train/verl/tools/statelm_tools.py, train/verl/experimental/agent_loop/statelm_agent_loop.py, train/verl/experimental/agent_loop/agent_loop.py, train/verl/trainer/ppo/ray_trainer.py). Its estimator declares the division of responsibility:

Partial rollouts and subtree-mean reward assignment happen in the agent-loop worker before this estimator runs. Consequently, each input row already represents one snapshot whose reward is the mean terminal reward of its descendant continuations. --- train/verl/trainer/contextpilot/adv.py

Tracked top-level coverage

The table accounts for every top-level tracked file or directory. Included means within the functional boundary, not that every contained implementation has been audited. The inspected material paths below establish the boundary's main loops; additional files at this same revision remain eligible evidence under SRC-1.

Frozen revision, dependencies and limits

The applicability cutoff is 2026-10-13, the supplied run date. The frozen boundary is commit 782cbb6611fb610c4cf6fafda6022b7e89cae191 of https://github.com/Tencent/ContextPilot, accessed read-only at the absolute checkout in source.path. HEAD matched that pin and the checkout reported no tracked modifications. The overall tier is code-grounded; reading source establishes implementation and declared contracts, not successful operation.

External dependencies include OpenAI-compatible inference and judge services, model/tokenizer weights, Elasticsearch, and the vLLM/verl training and serving stack. ContextPilot's shipped adapters, launchers and modified verl code are inside the boundary; separately installed dependency internals, deployed services and downloaded weights are not frozen evidence. Accordingly this run cannot establish provider execution semantics, model quality, service reliability, hardware performance or the exact behavior of an unpinned deployed dependency (SRC-1, infer/scripts/run_full_pipeline.sh, infer/scripts/eval_task.sh, train/README.md, .env.example).

Benchmark code and references are included, but actual task content is not uniformly available. infer/data/LongMemEval/longmemeval_s_cleaned.json is an LFS pointer in this checkout, and the inspected BrowseComp+ shard is likewise a pointer; infer/.gitattributes declares both data families as LFS. Their payloads were not fetched. NovelQA's prepared copyrighted bundle and externally downloaded InfBench and transformed LongBench-v2 training data are not supplied frozen inputs. These gaps prevent inspection of those complete datasets, reconstruction of the paper's exact evaluation inputs and independent benchmark verification, not characterization of the shipped data-loading and evaluation mechanism (SRC-1, infer/.gitattributes, infer/data/LongMemEval/longmemeval_s_cleaned.json, infer/data/BrowseCompPlus/data/test-00000-of-00006.parquet, infer/scripts/eval_task.sh, train/README.md).

No target code, example, training job or benchmark was executed. No observed run or causal experiment is registered. The linked paper, live demo, external StateLM repository and model collection were not acquired and are excluded as independent evidence; their contents cannot support conclusions in this run. Source-native documentation may report outcomes, but such reports do not establish reproduced gains, action-level causal effects, cross-session memory durability, or robustness under real deployment (SRC-1, README.md). Available files not yet inspected are not access gaps.

Source register