Halo synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From captured executions to an investigation
Halo is a harness-improvement plane built around a model-directed trace-analysis runtime. It takes captured executions, investigates failure patterns, retains diagnostic text and offers a report to a separate coding actor. This account covers the repository at commit b7f8509745d67b499b4e80efe20ea37c03426a74, with applicability cutoff 2026-10-14. The shipped desktop, engine, demo callers, evaluation example and improvement instructions are included. Receiving coding-agent execution, arbitrary harness edits, deployment and subsequent production collection are outside the frozen boundary. The classification is therefore complete artifact, partial loop, not an automatically closed improvement system. SRC-1 supplies inspected implementation, declared procedure and attributed benchmark results; no target execution or causal comparison was inspected. See boundary and evidence and the Source register.
The representative workflow starts when a desktop user asks a diagnostic question over selected stored traces. Host code owns scheduling, engine readiness and telemetry export. It rejects an empty export before launching analysis. A Python bridge passes the exported JSONL, question/history, provider settings and resource choices to the engine. This connected path was selected because it reaches retained delivery and handoff, rather than joining unrelated capabilities. CLI and library callers reach the same core engine but can grant optional repository access that this desktop bridge does not grant. See RT-RTE-desktop-analysis, RT-RTE-trace-diagnosis and workflow account.
Retained evidence has several distinct forms. SQLite holds normalized telemetry; JSONL exports preserve payload-bearing attributes; sidecar files hold derived offsets, aggregates and freshness metadata. The indexes locate evidence rather than replacing it with learned diagnoses. Their reuse predicate checks schema, file size and mtime, not cryptographic content identity. Existing desktop exports are reused by path existence, so a follow-up need not see newly ingested telemetry. Initial trace authenticity and adapter-specific mapping fidelity remain unresolved. See MEM-OBJ-traces-jsonl, MEM-OBJ-trace-index and EPI-RTE-index-reuse.
Who decides, and what reaches the model
The root model owns investigation choices: next read, delegation, interpretation and answer. Code owns tool dispatch, local state and event delivery. A child starts with its own system prompt and the parent's focused question, not the parent's transcript. It shares the trace store and configured access, then returns answer text, identity, sequence bounds and usage counters. Host-visible child events are a separate channel; the parent does not automatically receive the child's original investigation. Child failures return through the same answer-shaped channel for the parent to interpret, without a separate correctness verifier. See RT-RTE-subagent-call.
Delegation guidance is not enforced division of labor: the root retains leaf tools. Depth enforcement prevents deeper child creation; semaphores bound active children separately at each depth, not globally. SDK turn limits apply per invocation. Retries and root final-answer recovery can start another invocation, while direct synthesis and compaction calls have different accounting. These controls establish bounded routes, not a whole-run cost ceiling. Retry reconstruction trims incomplete tool tails but cannot retract already emitted events, so exactly-once effects are not guaranteed. See RT-CLM-delegation-policy and RT-RTE-trace-diagnosis.
Evidence access is model-requested pull through IDs, filters and raw regex searches. Returned views truncate attributes and can omit all spans when oversized. Raw search can recover matching windows, not ensure exhaustive relevant evidence. The separate synthesis model receives reduced metadata rather than full raw payloads; an oversized trace can render as spans: 0 without its omission explanation. A computed trace count is cardinality under the selected index/filter premises, not task success or a causal failure explanation. See MEM-RTE-trace-read, RT-RTE-trace-synthesis and EPI-RTE-count-traces.
Optional Python execution and read-only code/Git tools extend investigation for appropriately configured callers. Deno grants and timeouts constrain the sandbox route; path confinement and fixed Git operations constrain repository APIs. Neither establishes whole-host isolation or deployed security. The skill's blanket claim that Halo cannot see code is stale for repository-enabled invocations, though it matches this desktop configuration. See RT-RTE-sandbox-python, RT-RTE-repository-inspection and RT-CLM-trace-only.
From an answer to later reliance
Root final_answer acknowledges a structured submission and stops the SDK loop; it does not test truth. The desktop uses a weaker completion rule: absent a final-flagged answer, it selects the last assistant item, not necessarily a root item. Nonempty text can therefore produce completed without successful root finalization. These are operational admission predicates, not evidence-consuming acceptance of a diagnostic claim. No concrete generated diagnosis was inspected, so whether its content preserves, derives or adds claims remains indeterminate. See EPI-RTE-answer-finalization, RT-RTE-desktop-analysis and EPI-OBJ-diagnostic-answer.
The strongest supported retained-context contribution is the wired chain from trace investigation to automatically stored diagnostic text to later desktop follow-up input. This is trace learning in the write-route sense: traces feed a durable behavior-shaping artifact with a later model consumer. It does not establish improved future capacity, parameter training or actual changed decisions. Earlier answers reach that consumer as conversational evidence, not standing instructions. A new question triggers automatic history supply, bounded by run identity and coarse turn/role/content selection rather than semantic relevance. The history builder includes earlier nonempty assistant text regardless of status and has no explicit token cap. Raw tool/child events are separately retained for human display, not replayed with those answers. See MEM-OBJ-halo-run-turns, MEM-BAP-followup-history and profile verification.
Optional checkpoint compaction is a different contribution. Only successful root completion with the checkpoint flag triggers it; ordinary CLI and desktop configurations leave it disabled. Code selects older units, a model summarizes them, and successful complete units replace rendered content without a semantic check. Local originals remain retrievable, but emitted replay messages do not carry the backing original-item index. A host can persist and re-supply summaries and tails; that durable continuation chain is afforded, not wired by this desktop. This is not continuous mid-analysis context management. See RT-RTE-checkpoint-emission, MEM-OBJ-agent-context and MEM-RTE-context-reload.
From retained findings to a proposed harness change
Report materialization deterministically projects retained conversation into Markdown. It includes user turns and nonempty completed/incomplete assistant turns, not the full event stream. Answer retention, report-file creation and external prompt initiation are separate delivery milestones; report-write failure can be retried through the UI without undoing retained text. A coding-tool link carries a report path and prefilled request. The user must confirm initiation inside the receiving tool. Neither file freshness nor confirmation validates findings or proves later reading, editing or deployment. See WF-RTE-report-materialization, MEM-OBJ-report and RT-RTE-report-handoff.
The shipped skill prescribes a substantive external procedure: establish a baseline, diagnose traces, verify claims against source, make a minimal hypothesis-driven edit, then compare evaluation delta and error-pattern frequency before shipping. The desktop link does not establish that the recipient loads that skill. AppWorld supplies a separate implemented evaluator over task outputs and reference data, with saved pass/fail reports. Its license is task-criterion satisfaction, conditional on task-module and data fidelity; it does not certify Halo's explanation, isolate component effects or automatically admit a patch. The README reports harness-only benchmark gains and source-trace verification, but the underlying paired runs and interventions were not inspected. The supported result is a wired diagnostic input to an improvement process, with attributed external gains—not independently established causal improvement by Halo. See EPI-RTE-task-evaluation, MEM-OBJ-evaluation-report and EPI-CLM-benchmark-improvement.
The independent property assessments remain separate:
| Property and scope | Supported assessment |
|---|---|
| Theory-builder condition 1: localized hypothesis | Claimed in the prescribed handoff procedure; no executed whole hypothesis inspected. |
| Condition 2: decisions depend on hypothesis content | Claimed through choosing an evidence-grounded edit; actual dependency uninspected. |
| Condition 3: content-directed criticism and revision | Claimed through failed remeasurement prompting sharper diagnosis; no episode inspected. |
| Condition 4: criticism shapes another round | Claimed as repeated diagnosis/remeasurement; retained criticism affecting a next round uninspected. |
| Learning from criticism of a consumed theory | Improved future capacity unestablished; retained answers alone do not demonstrate it. |
| Reflection | Trace representation supports proposed changes to a selected external harness, but Halo's own representation-mediated change loop is unestablished. Reflective theory building is uninspected. |
| Autonomy | Autonomous theory building is uninspected; confirmation and external coding execution remain consequential, and computation performing every internal operation is not established. |
| Exercised self-improvement of Halo | Unestablished over subsequent diagnosis sessions toward better trace-grounded diagnosis. Retained context is a positive contribution, but objective-bearing evidence shaping an organizational update and causally dependent later operation have not been demonstrated. |
These assessments follow RT-RTE-report-handoff, with the retained-context contribution independently supported by MEM-BAP-followup-history. Host installation updates Halo machinery through Git/uv and an importability gate, not through trace-responsive evaluation; it supplies no missing improvement attribution (RT-RTE-engine-install).
For a scenario requiring exploratory diagnosis and conversational follow-up, the implemented architecture connects evidence access, optional focused delegation and retained delivery. For a scenario requiring verified causal findings or automatically deployed improvements, those same milestones are insufficient. The assessment would change with inspectable runs linking source evidence to specific diagnoses and later decisions, retained claim verification and criticism affecting subsequent rounds, and paired interventions measuring the stated objective at the appropriate treatment level. A wired checkpoint consumer would strengthen continuation claims; content-fidelity checks would strengthen summary claims; an evidence-responsive update of Halo's own organization followed by causally dependent operation would change its self-improvement assessment. None follows from the present delivery labels.
Limitations
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| No inspected execution or intervention comparison | SRC-1; RT-RTE-trace-diagnosis, EPI-CLM-benchmark-improvement | Frozen code, procedure and attributed README results; no target execution | Exercised diagnosis, reliable compliance, observed benefit and causal benchmark attribution | Linked run artifacts, specific findings/edits and paired interventions with declared controls and treatment scope |
| External improvement loop remains open | SRC-1; RT-RTE-report-handoff, RT-RTE-engine-install | Report-path initiation and prescribed procedure; external coding/deployment excluded | Autonomous loop closure, performed theory-builder conditions, learning through criticism and exercised Halo self-improvement | Retained hypothesis/criticism/update episodes, receiving execution and later operation linked to the bounded system's own objective and organization |
| Input authenticity, mapping and cache identity are incomplete | SRC-1; MEM-RTE-telemetry-admission, EPI-OBJ-span-record, EPI-RTE-index-reuse | Common ingestion/export and stat/schema reuse inspected; adapter internals and arbitrary input provenance unresolved | Authenticated events, complete end-to-end fidelity and content-identical cache reuse | Producer provenance, adapter mapping checks and content-identity/concurrency evidence |
| Bounded views and reduced synthesis inputs omit evidence | SRC-1; MEM-RTE-trace-read, RT-RTE-trace-synthesis, EPI-RTE-count-traces | Inspected caps, rendering and count; full filter/accumulator audit not established | Exhaustive relevance, complete synthesis evidence and real-world success-frequency warrant | Complete relevant payload access, propagated omission metadata and checked index/filter premises |
| Completion and persistence do not provide semantic acceptance | SRC-1; EPI-RTE-answer-finalization, MEM-OBJ-halo-run-turns, MEM-OBJ-report | Protocol acknowledgement, bridge fallback, history and report selectors | Certified diagnosis truth, root provenance from completed status, or full supporting investigation in later context | Concrete candidate/evidence pairs, scoped evidence-consuming acceptance and provenance-preserving later delivery |
| Checkpoint preservation and durable replay remain qualified | SRC-1; RT-RTE-checkpoint-emission, MEM-OBJ-agent-context, MEM-RTE-context-reload | Terminal flag-gated compaction and local reload; desktop does not consume checkpoints | Continuous compaction, faithful summary guarantees, wired durable resume and recovery of disposed originals | Semantic fidelity checks plus a retained checkpoint consumer and backing-original reconstruction policy |
| Cleanup and refresh are not universal withdrawal | SRC-1; MEM-RTE-retention-cleanup, MEM-OBJ-traces-jsonl, MEM-OBJ-halo-run-turns | Separate telemetry/run stores, best-effort files, export reuse and retained retry content | Guaranteed physical erasure, propagation to prior recipients or uniformly fresh/valid follow-up evidence | Exercised cross-store withdrawal and refresh semantics, including copied reports and concurrent readers |
| Provider state and deployed containment are uninspected | SRC-1; RT-CMP-root-agent, RT-CMP-subagent, RT-CMP-synthesis-model, RT-CMP-compaction-model, RT-RTE-sandbox-python, RT-RTE-repository-inspection | Local inference bindings and route restrictions, not external implementations | Complete acquired-memory inventory, immutable provider behavior, parameter-learning conclusions or deployed security guarantees | Provider-state/version evidence and deployment-specific containment/security checks |
| Caller continuations and arbitrary payloads remain unresolved | SRC-1; MEM-OBJ-agent-context, MEM-OBJ-traces-jsonl, MEM-OBJ-halo-run-events; profile verification | Known local paths with unspecified initial caller control and mixed payload consumers | Complete provenance, form, authority and admission classifications across included alternatives | Identified content/admission mechanisms and actual consumer semantics for each unresolved part |
| Task-specific evaluation warrant is bounded | SRC-1; EPI-RTE-task-evaluation, MEM-OBJ-evaluation-report | Evaluator dispatch/tracker inspected; task modules, data and aggregation not fully inspected | Benchmark reproducibility, complete oracle fidelity, mechanism truth or automatic score-to-policy revision | Task-module/data audit, retained evaluations and a separately evidenced consequential update consumer |
| Evaluator inventory omits a standalone trace-conformance checker | SRC-1; report verification Limits, EPI-RTE-task-evaluation, EPI-OBJ-span-record, EPI-RTE-answer-finalization | Verification inspected demo/openai-agents-sdk-demo/verify_traces.py, separately invoked by demo/appworld/Taskfile.yml; analyst inventory omits it |
Exhaustive shipped-evaluator and input-validation comparison; checker assertions do not authenticate events or validate answers | Canonical records covering that conformance route and its consumers; execution evidence for any operation claim |