Ingest: Autogenesis: A Self-Evolving Agent Protocol

Type: kb/sources/types/ingest-report.md

Source: autogenesis-a-self-evolving-agent-protocol.md Captured: 2026-08-02 From: https://arxiv.org/html/2604.15034v5

Classification

Genre: scientific-paper -- an arXiv v5 preprint that specifies a self-evolution protocol, supplies an implementation and project repository, and reports benchmark comparisons across reasoning, general-agent, and code tasks. Domains: self-improvement, agent-protocols, deploy-time-learning, evaluation Author: A seven-author team from Nanyang Technological University, Stanford University, City University of Hong Kong, and the University of Science and Technology of China. The released project code raises inspectability, while the authors' dual role as system builders and evaluators makes independent reproduction important.

Summary

Autogenesis Protocol (AGP) separates a Resource Substrate Protocol Layer from a Self-Evolution Protocol Layer. The substrate registers prompts, agents, tools, environments, and memory as typed, versioned resources with lifecycle, retrieval, execution, serialization, and rollback operations; the evolution layer lifts selected resources into a trainable subspace and composes operators that observe traces, infer failures, propose changes, evaluate candidates, and commit or reject them. Its concrete Autogenesis System coordinates a planning agent and specialist sub-agents over a message bus and applies reflection-driven edits during execution. Across GPQA, AIME, GAIA, HLE, and a 100-problem multilingual LeetCode set, the paper reports improvements from prompt, solution, joint prompt-solution, or agent evolution, but the comparisons primarily show that additional bounded refinement can improve the tested configurations; they do not isolate the resource ontology, operator protocol, versioning layer, or safety machinery as causes.

Connections Found

The source is a technical implementation case for the proposal-selection improvement loop: Reflect/Select/Improve supply search, Evaluate can reject, and Commit plus version lineage makes accepted changes operative. Its versioned prompt, tool, and agent-code edits add a mixed-form case for the readable-artifact loop, while the trace-to-hypothesis step depends on diagnostic richness without testing alternative evidence surfaces. Most distinctively, the binary learnability mask gives learning inside a fixed decomposition an unusually explicit worked case: the experiments improve mappings expressible through selected resource edits while leaving the ontology, orchestration, evaluator, and benchmark interface fixed. At the system level, AGP compares with Exo: both protect versioned state beneath mutable agent machinery, but Exo is a code-grounded live self-rebuild harness whereas AGP presents a typed protocol and narrower benchmarked edits. It also follows the earlier whole-readable-harness direction in Symbolic Learning, replacing language-mediated backward credit assignment with lifecycle management and an optimizer-independent operator interface.

Extractable Value

  1. The learnability mask turns the effective update boundary into inspectable protocol state. Each resource variable carries an explicit editable/frozen marker, making it possible to audit what an improvement claim could and could not have changed rather than inferring the boundary from an implementation after the fact. [quick-win]
  2. The protocol separates heterogeneous retained artifacts from the optimizers acting on them. Registration records, context managers, version lineage, and uniform lifecycle operations provide a concrete design for applying reflection, textual critique, or reward-driven methods to prompts, tools, agents, environments, and memory without embedding optimizer logic in each resource. This is a design contribution, not evidence that the five types are exhaustive or best. [deep-dive]
  3. Reflect/Select/Improve/Evaluate/Commit is a directly typed proposal-selection architecture. The operator signatures distinguish diagnostic evidence, hypotheses, modifications, objectives, evaluation results, and committed state, giving the KB a cleaner system instance than loops whose search, gate, and retention must be reconstructed from prose. [quick-win]
  4. Version lineage and rollback solve state reversibility, not semantic safety. Immutable versions and conditional commit make a rejected state recoverable and auditable, but the acceptance oracle still determines whether a harmful or merely flattering change is retained. This sharp separation is useful when evaluating claims of safe self-modification. [quick-win]
  5. The strongest empirical result is execution-guided solution repair inside a fixed interface. With one Gemini backbone and three reflection rounds, Solution-Evo raises accepted solutions by 10.1--26.7% across five programming languages and reduces several execution-error categories. The executable judge supplies rich, actionable feedback, but the result supports bounded code revision more strongly than the general protocol. [experiment]
  6. Prompt and solution edits can be complementary, but the evidence is context-bound. Joint evolution often improves AIME and GPQA over the vanilla run, yet it does not uniformly beat both single-target variants, and the 30-item AIME sets make large relative percentages unstable. Treat this as a candidate interaction between readable targets, not an established rule to evolve both. [just-a-reference]

Limitations (our opinion)

The central comparisons add up to three inference-time refinement rounds to a one-pass vanilla baseline without an equal-compute best-of-n, repeated-sampling, or generic self-refine control. The results therefore do not isolate protocol-governed evolution from extra test-time computation. Several samples are small (AIME has 30 problems per year), yet the paper reports large relative gains without confidence intervals, significance tests, or repeated-run variance. GAIA/HLE methodology is not detailed enough to establish how candidate evaluation, cross-task reuse, task ordering, and benchmark independence interact, and HLE relies on an LLM judge.

Under the fixed-decomposition lens, the reflection optimizer can condition behavior on current resource state, reasoning and tool traces, execution failures, benchmark feedback, and objective/safety fields. It can compose typed resource operations and LLM-authored edits to prompts, solutions, agent code, and tools within the enabled learnability mask. Its hypothesis class is the configured proposer/editor models plus those declared edit surfaces. Fixed outside that space are the five-entity ontology, variable-lifting representation, agent bus and specialist roles, trace schema, evaluator, acceptance rule, model backbones, task interfaces, benchmark distribution, and three-round budget. Prompt-Evo, Solution-Evo, and joint variants vary the editable target; they do not validate adjacent fixed choices or the decomposition as a whole. Environment and Memory evolution are claimed as implemented but are not independently evaluated.

The phrase "safe-by-construction" overreaches the evidence. Versioning and rollback establish mechanical traceability and reversibility, while the paper neither specifies substantial safety invariants nor measures behavioral drift, alignment failures, adversarial updates, or the reliability of LLM-generated causal diagnoses. A bad candidate accepted by a weak oracle remains versioned bad behavior. Reproducibility also needs code inspection: the appendix says the supplied code dataset covers JavaScript where the main experiment reports Kotlin, and the broad protocol comparison assigns capability and complexity values to AGP, A2A, and MCP without an independent evaluation of those claims.

Write kb/agentic-systems/autogenesis.md as a code-grounded analysis pinned to a repository commit, verifying whether the implementation actually enforces version lineage, rollback, learnability masks, safety gates, and Memory/Environment evolution before treating the paper's protocol claims as shipped system behavior.