PrimeScientist boundary

Type: agentic-system-analyses/types/agentic-system-boundary.md

Boundary and evidence

This analysis covers PrimeScientist as a model-dependent research-search workflow, not as the implementation of its underlying coding-agent runtime. Its intended use is budgeted research-paper replication and code optimization. The shipped driver initializes plans through a reflector, selects filesystem-backed tree nodes, launches an agent through a benchmark adapter, evaluates results, retains artifacts and scores, and uses them for subsequent selection and proposals. The boundary includes both tree search and the shipped linear baseline. SRC-1, README.md, run_search.py, run_vanilla.py, search/tree.py, search/reflector.py.

The target's own description establishes the intended division of labor:

Code for PrimeScientist, a budget-adaptive tree-search framework for autonomous research agents. Each tree node is an experimental plan executed by a coding agent (Codex); a reflector agent reads each result and proposes child plans. --- README.md

The shipped driver is part of the target, not an excluded application using a library. It owns state reloading, budget checks, selection, benchmark calls, score admission, pruning and reflection. The benchmark adapters and their runners are likewise included: they stage retained plans for agent consumption, create or inherit sandboxes and return evaluation packets. The following executable call sequence grounds the workflow classification. SRC-1, run_search.py:

    run_id = benchmark.run_agent(task, agent, model, run_env,
                                 env_override=env_override,
                                 node_dir=node.path)
    if run_id is None:
        continue
    packet  = benchmark.evaluate(task, run_id)
    primary = benchmark.primary_score(packet)

--- run_search.py

The boundary is whole-system for the shipped PrimeScientist workflow. It includes state production and persistence in tree files and ledgers, plan and workspace inheritance, reflector prompt templates and session bookkeeping, later state reload and consumption, benchmark evaluators, task assets and launchers. This is not a claim that every task dataset or vendored evaluator internal has already been inspected. The initial inspection traces the material control paths; the coverage table distinguishes included but not-yet-inspected supporting material from unavailable evidence. SRC-1, search/tree.py, search/reflector.py, run_search.py, run_vanilla.py, benchmarks/base.py, benchmarks/registry.py.

Filesystem state is substantively inside the boundary. Tree nodes persist plans, packets, statistics and priors; the driver resumes an existing tree. The baseline reloads prior packets and stages history before further agent calls. FIRE-Bench runners consume staged task plans and any available cross-task insights, and the inheritance runner copies a parent sandbox before replacing its staged plan. These are inspected consumer paths, not merely declarations in an API. Their presence does not establish that cross-task insight production forms a functioning closed loop. SRC-1, search/tree.py, run_search.py, run_vanilla.py, benchmarks/fire_bench/agents/codex/run.py, benchmarks/fire_bench/agents/codex/run_inherit.py.

root_path = tree_dir / "root"
if root_path.exists() and (root_path / STATS_FILE).exists():
    return TreeNode(root_path, task)
return None

--- search/tree.py

The shipped benchmark integrations are AutoLab and FIRE-Bench. AutoLab stages a plan, invokes Codex in a sandbox and runs task verification through Docker or enroot. FIRE-Bench dispatches agent runners and invokes the vendored RAGChecker evaluation entry point; its inspected evaluation path derives a conclusion from a log and uses model-backed extraction/checking. OpenHands also has a shipped FIRE-Bench runner and is included as an alternate integration, despite the README's narrower supported-backend statement. Its prompt injection and Docker dispatch were inspected; operational compatibility is not established. SRC-1, benchmarks/autolab/__init__.py, benchmarks/fire_bench/__init__.py, benchmarks/fire_bench/run_agent.py, benchmarks/fire_bench/agents/openhands/run.py, benchmarks/fire_bench/eval/RAGChecker/eval.py, README.md.

The registry distinguishes shipped integrations from an unavailable optional integration:

Shipped in this repo. terminal_bench is dispatched but not vendored here, so

it is listed separately and only reported as available once importable.

_SHIPPED = ("fire_bench", "autolab") --- benchmarks/registry.py

External dependencies include the Codex CLI and model service, optional OpenHands runtime images, container runtimes, host launch tooling, Python packages and task-specific model/data services. The target includes invocation, prompt/context injection and output handling, not those external runtimes' internal scheduling, tool policy or context management. Their implementation is not frozen here; conclusions about their internal guarantees are prevented. SRC-1, search/reflector.py, benchmarks/autolab/__init__.py, benchmarks/fire_bench/agents/codex/run.py, benchmarks/fire_bench/agents/openhands/run.py, README.md.

No other repository, paper linked from the README, live webpage or later revision supplies evidence. The frozen checkout is at 29971beac6f4f4b41309b1326762e1b83ceece98, with an applicability cutoff of 2026-10-14. HEAD matched the acquired revision and the checkout status was clean. The evidence tier is code-grounded: source inspection supports executable paths, while documentation supplies attributed design intent. No target execution, benchmark run or causal experiment was performed. The README reports benchmark evaluation, including MLE-Bench, but the inspected registry does not provide an MLE-Bench integration. This prevents treating that reported evaluation as a shipped, inspected execution path or as observed performance. SRC-1, README.md, benchmarks/registry.py.

Top-level coverage

All rows refer to SRC-1 at the frozen commit. “Not yet inspected” is an inspection limit, not an access gap or exclusion from the registered repository.

Source register