Hyperagents synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
Hyperagents implements an improvement plane: a host driver generates code-modified agents, evaluates them, retains their lineage and results, and selects code for another generation. Its consequential output can change the editor and tools that produce future agents, not only task answers. This account covers the whole shipped framework at Git commit 59a68f672dfb92c74aeb7e61535d776fb36e172d, with an applicability cutoff of 2026-10-14. SRC-1 supplies inspected implementation and declared design only. No target execution, experiment logs or causal comparison was frozen; the paper, blog and external dependency implementations are outside this evidence. Thus the account establishes implemented pathways, not successful runs or measured gains (boundary and evidence; RT-RTE-generation-cycle).
From a generation request to executable change
An operator chooses domains, budget and modes. The host owns scheduling, generation identities, root preparation and resume. It reconstructs a selected parent by replaying ordered saved patches into a writable repository, then launches the inherited editor. The initial editor starts a fresh conversation and asks a provider model to modify the codebase. Python parses textual tool calls, dispatches registered tools and limits their count; the model chooses edits. Generated descendants can change task code, editor code, tools and optional selector or ensemble logic. Provider inference does not itself update the provider model's weights (RT-RTE-editor-admission, RT-RTE-model-tools, RT-CMP-provider-models).
The archive has two importantly different effects. First, saved diffs become executable code through later replay and Python imports. This is stronger than advisory historical context. Second, previous evaluations and logs can become model-readable evidence. Normal delivery copies a filename-filtered archive view, removing current-generation, validation/test-named and repository-copy material. The initial editor accepts an evaluation path but does not insert it or its contents into the prompt; historical evidence use requires discovery and a tool read. Parent-only no_archive copying is unpruned, while optional selector/ensemble wrappers receive a full archive. These are different visibility policies, not proof that held-out information is semantically isolated (RT-BAP-code-replay, MEM-RTE-archive-delivery, RT-RTE-optional-consumers).
Context control also operates inside each invocation. Initial editor and task conversations start empty. Task workers return predictions while logs persist separately. File-view results can be clipped, with disk contents still available for another range request. The model helper accumulates turns; its output-context retry is not a demonstrated summarization mechanism. Fresh task frames therefore limit conversational carryover without making execution a pure return-only boundary: filesystem effects and imported code can escape it (MEM-OBJ-chat-history; memory annotations).
From edited code to scores and another parent
The representative workflow is one paper-review generation. After editing, the runner attempts to restore domains/ and saves a diff. A nonempty diff and successful imports admit an evaluation attempt, not an improved or trustworthy agent. The harness creates a fresh task agent for each paper, supplies paper text without its outcome label, and saves predictions. The report normalizes strings and measures agreement with dataset labels. Positive staged scores for every selected domain allow fuller evaluation unless staging is skipped. Full evaluation extends existing prediction tables; it does not necessarily rerun completed rows (RT-RTE-task-evaluation, MEM-RTE-prediction-reuse; workflow account).
These checks have bounded targets. Label agreement supports a conditional statistic over supplied rows, not independent truth of the labels. Proof grading delegates judgment to an external grader before a fixed points conversion; it is not inspected formal verification. BALROG checks actions and summarizes environment progress. Genesis evaluates trained policies through fitness metrics. Polyglot attempts patch tests but has a logger-call defect on its nonempty-patch path. None of these outcomes by itself explains why an editor change works or licenses transfer (EPI-RTE-label-scoring, EPI-RTE-proof-grading, EPI-RTE-outcome-summary, RT-RTE-domain-effects).
On normal return to the outer loop, a generation is archived even when it is ineligible. Separate flags and scores govern successor selection. Starting-node exceptions, first-node fallback and eligibility's meta_patch_files is not None alternative prevent a universal successful-evaluation prerequisite; the CLI's empty-list default also satisfies that alternative. Parent selection chooses an exploration lineage, not necessarily an agent better than its parent. Failure can invalidate parent or selector reuse without deleting its retained history (RT-OBJ-metadata, RT-RTE-parent-selection, RT-BAP-selector-flags).
Optional consumers alter this progression. Editable selection runs code from the latest eligible selector source, which need not be the task-performance parent; exceptions disable that source and trigger bounded retries. The initial ensemble selects an archived agent using the requested split's score and returns its saved prediction, rather than making a new task-model call. Validation/test-score-conditioned selection cannot establish unseen-split generalization (RT-RTE-optional-consumers).
Delivery is a directory containing available diffs, logs, predictions, reports and metadata, plus a lineage that can be replayed later—not a deployed winning agent. Cleanup is best effort; outer plotting and selection are outside the local generation exception handler. Docker's writable mount and host networking, post-edit domain restoration and mutable dependencies do not establish adversarial containment or evaluator integrity (workflow delivery, RT-RTE-editor-admission, RT-CLM-bash-description).
Learning and improvement: positive pathways, separate unresolved properties
The strongest supported trace-fed contributions are explicit, consequential update routes. In optional DGM mode, selected failed-case reports and interaction or episode traces feed model diagnosis; extracted implementation/problem fields reach a later coding agent and can lead to retained executable changes. Genesis separately wires rollout-fed numerical policy training to saved checkpoints and later action inference. These are implemented trace-fed behavior-shaping routes. Ordinary archive-informed editing remains an affordance because copied evidence is not automatically consumed. No comparison establishes improved future capacity from any of these routes (MEM-RTE-dgm-diagnosis, RT-CMP-policy-runner, MEM-RTE-archive-delivery).
The records do not establish a theory-builder assessment for the system. Condition 1, localized formulated theory, is not established merely by an executable patch or opaque parameters. Condition 2, decisions depending on that theory's content, is not established by archive availability. Condition 3, criticism identifying a challenged claim, result and placement of blame, is not established by scores or the diagnosis label. Condition 4, retaining criticism's result to shape another round, is not established merely by replaying code. These are separate unresolved findings, not evidenced absences; the DGM route and opaque model processing prevent a universal negative. Improved capacity specifically caused by criticism of a consumed theory is likewise unestablished (RT-RTE-editor-admission, MEM-RTE-dgm-diagnosis; epistemic conclusion).
Reflection is not established by editing one's generating machinery alone: the accepted account does not establish the required self-representation and both causal directions. Computational scheduling and automatic per-change admission are implemented, but they do not establish an autonomous theory builder when its theory operations remain unresolved. These qualifications do not erase the supported automatic edit, evaluation and replay mechanisms (RT-RTE-generation-cycle, RT-RTE-editor-admission).
Self-improvement has a stronger architectural finding: within later generations of the same host run, an independently specified task-performance objective and a live change-to-own-machinery channel support dispositional improvement-directed self-change. Exercised evidence-responsive self-improvement remains unestablished because no trace shows objective-relevant evidence determining the actual update or subsequent operation causally depending on it. Successful improvement requires separate outcome evidence. There was no treatment/comparison here, so no system-level gain or component effect can be attributed (RT-CLM-recursive-improvement).
For bounded benchmark search, the architecture connects executable variation, outcome measurement and lineage reuse. For an account of warranted mechanism discovery or demonstrated future-capacity improvement, its evidence stops earlier. A revision-linked trace showing selected evidence, update determination, replay and dependent later behavior would change the exercised-self-improvement assessment. Comparable before/after outcomes would address benefit; attributing a component effect would require independently varying that component. Reliable recovery claims also require repairing the identified branches and observing them, while stronger evaluation-integrity claims require controls beyond mutable path conventions (RT-CLM-recursive-improvement; report verification).
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No execution or causal comparison was frozen | SRC-1; RT-CLM-recursive-improvement, RT-RTE-generation-cycle | Pinned implementation and design, not external experiment logs | Actual evidence use, exercised self-improvement, dependable operation, improved capacity or causal component benefit | Revision-linked execution artifacts, update-dependence traces and appropriately controlled outcome comparisons |
| DGM diagnosis retry omits a required argument | SRC-1; MEM-RTE-dgm-diagnosis, RT-RTE-editor-admission; report limit | Diagnosis extraction and recursive retry in baselines/dgm/utils.py |
Dependable retry/recovery or successful diagnosis-driven revision; initial architectural branch remains supported | Corrected retry implementation and traces exercising extraction failure, recovery and subsequent revision |
| Polyglot nonempty-patch path passes an unsupported logger keyword | SRC-1; RT-RTE-domain-effects, MEM-OBJ-episode-log; report limit | domains/polyglot/harness.py call to shared logger in utils/docker_utils.py |
Reliable resolved/unresolved classification and delivery on that unchanged branch; general correctness also exceeds test scope | Compatible logger invocation and branch execution artifacts; broader solution claims need broader checks |
| Domain artifacts are not complete provenance or read-back inventories | SRC-1; MEM-OBJ-report, MEM-OBJ-episode-log, RT-RTE-domain-effects; epistemic coverage, report limit | Detailed environment-context construction, checkpoint internals and statistical/transfer consumers incompletely characterized; transfer helper selects candidates and emits replay commands | Exhaustive route comparisons, complete trace provenance, reconstructed numerical-policy state or transfer benefit | Recorded per-environment producers/consumers, checkpoint/dependency inspection and actual transfer evaluations with suitable comparisons |
| Grouped comparison units are not per-part enumerations | SRC-1; MEM-OBJ-predictions, MEM-OBJ-chat-history, MEM-OBJ-episode-log, MEM-RTE-dgm-diagnosis, RT-RTE-optional-consumers; profile limit | Grouped form, authority and admission classifications with partial coverage | Per-part counts, complete route-by-route comparisons or assigning every grouped finding to every named object/consumer; local existence findings stand | Disaggregated accepted records and units naming each part, admission mechanism and actual consumer |
| Initial provenance and external validity remain unresolved | SRC-1; MEM-OBJ-report, EPI-RTE-label-scoring, EPI-RTE-proof-grading, RT-CMP-policy-runner | Bootstrap approval, reference origins, external grader/training/environment implementations and checkpoint initialization | Trustworthy initial-report provenance, independent label truth, formal proof soundness or full training reconstruction | Authenticated provenance and inspected grader, reference and training evidence at matched versions |
| Provider internals and some retained payloads remain opaque | SRC-1; RT-CMP-provider-models, RT-CMP-policy-runner, MEM-OBJ-episode-log | Provider inference boundary, auxiliary checkpoint fields and retained images without an established operative consumer | Complete memory-form, lineage, authority or hidden-learning inventory; neither absence nor benefit follows | Provider-side retention/update evidence, checkpoint payload/consumer specifications and identified image-consumption routes |
| Operational checks are not integrity or improvement acceptance guarantees | SRC-1; RT-RTE-editor-admission, RT-RTE-parent-selection, RT-RTE-optional-consumers | Mutable dependencies, post-edit reset, direct entries, eligibility exceptions, cache reuse and split-conditioned ensemble selection | Universal successful-score admission, evaluator independence, code-version-safe reuse, containment or unseen-split generalization | Explicit protected evaluation boundary, version-aware reuse, tightened admissions and independent held-out execution/comparison evidence |