LHTB boundary

Type: agentic-system-analyses/types/agentic-system-boundary.md

Boundary and evidence

LHTB is in scope as a workflow for evaluating model-driven terminal work, including its bundled execution and continuation machinery. The intended use of this analysis is to explain how task instructions, persistent sandbox artifacts, model context, hidden verification and run results interact. It is not a review of the scientific correctness of every benchmark problem or a replication of the leaderboard. The boundary is whole-system for that evaluation workflow, not for every independent use of the vendored Harbor framework.

The target includes both task generations, shipped launch configurations and drivers, the modified Harbor job/trial orchestrator, Terminus-2 and its context management, supported agent and environment integration paths, verifier isolation and artifact transfer, result persistence and later consumption, and the shipped patch-maintenance instructions. The bundled runtime is not an external dependency merely because the benchmark invokes its CLI. Task-local RAG, memory and simulation programs are benchmark workloads, not automatically general capabilities of the evaluation harness. These distinctions follow the shipped driver and consumers: SRC-1, configs/examples/full_benchmark_2.0.yaml, scripts/daytona_cleanup/run_daytona_eval.sh, harbor/src/harbor/cli/jobs.py, harbor/src/harbor/trial/trial.py, and tasks/langchain-version-migration/task.toml.

The bundled Harbor README identifies its connection to LHTB (SRC-1):

This copy of Harbor is not stock upstream Harbor. It is the exact harness used to run LHTB, based on Harbor 0.7.0 with the continue-until-timeout extension plus verifier-isolation and binary-feedback hardening. --- harbor/README.md

The default full-suite configuration selects a model-calling agent and both task roots:

agents: - name: terminus-2 model_name: openai/gpt-4.1 --- configs/examples/full_benchmark_2.0.yaml

datasets: - path: ./tasks - path: ./tasks-2.0 --- configs/examples/full_benchmark_2.0.yaml

Entry-point and state-loop coverage

Initial tracing followed the console entry point declared in harbor/pyproject.toml through harbor/src/harbor/cli/main.py and the job-start consumer in harbor/src/harbor/cli/jobs.py. That consumer creates and runs a Job. harbor/src/harbor/job.py expands task/agent configurations, reloads existing trial results, and reconciles remaining trials. The trial layer invokes the selected agent, runs verification, transfers configured artifacts to separate verifier environments when selected, and writes the final trial result. Terminus-2 constructs prompts from task instructions and terminal state, calls the model through Chat, and exposes a same-conversation continuation path. Thus this is not just a passive task dataset or a returning numeric grader. All paths in this paragraph are SRC-1 implementation evidence: harbor/src/harbor/agents/terminus_2/terminus_2.py, harbor/src/harbor/llms/chat.py, harbor/src/harbor/verifier/verifier.py, and the entry-point paths above.

The model-call boundary is explicit:

llm_response: LLMResponse = await self._model.call( prompt=prompt, message_history=self._messages, logging_path=logging_path, previous_response_id=self._last_response_id, **kwargs, ) --- harbor/src/harbor/llms/chat.py

The continuation consumer can reuse the existing conversation and terminal session:

async def resume_after_verifier_rejection( self, user_prompt: str, context: AgentContext, ) -> None: """Continue the existing Chat and tmux session after binary rejection.""" --- harbor/src/harbor/agents/terminus_2/terminus_2.py

Host-side result persistence is also part of the target, not an excluded caller:

self._trial_paths.result_path.write_text(self.result.model_dump_json(indent=4)) --- harbor/src/harbor/trial/trial.py

The coverage table accounts for every top-level tracked entry. “Included” fixes functional scope; it does not assert that every file in that subtree has already been read. Initial inspection is named separately from source that remains available for the downstream analysis. No uninspected file is described as inaccessible.

Frozen evidence and dependencies

The sole evidence source is SRC-1 at full Git revision 7ca0cacea9bf6a0694eaad7306bed794eaaaabce; the checkout HEAD was verified against the acquired pin and its status was clean. The applicability cutoff is 2026-10-15. Later commits, mutable web pages, the linked paper, upstream Harbor revisions, hosted datasets and unbundled execution traces are outside the frozen source set. The overall tier is code-grounded because the entry points and model-dependent continuation/verification consumers are inspectable, not merely described in documentation.

External dependencies include model services, sandbox/container providers, installed agent CLIs, task-specific scientific packages, and container images. Their configuration and invocation code in SRC-1 are included; their independently deployed implementation and service behavior are not. In particular, a digest-pinned image in tasks-2.0/acoustic_beamforming_peak__01/task.toml identifies a dependency but is not a captured image. These limits prevent an end-to-end runtime assurance, a claim that hidden material was unreachable in actual runs, or a reproducibility claim about live services. No target code, tests, examples, providers or environments were executed.

The README and assets/make_figures_2.0.py supply attributed result summaries and their declared plotting/data-consumption contract. They do not supply an independently inspected causal experiment. The actual leaderboard trajectories linked outside this repository were not acquired, so this boundary does not establish observed model behavior or causal benefits of continuation, summarization or verifier isolation. Available but uninspected source files remain under the same SRC-1 pin; inspecting them later does not require a new repository or revision.

Source register