Autogenesis

Type: kb/types/note.md · Tags: computational-model, deploy-time-learning, self-improving-systems

Evidence basis: first-hand reading of the Autogenesis paper v5, its paper-aligned v1.0.0 code at commit fd6ab4c, and the rewritten main at commit 5b070d5 (captured 2026-08-02). I ran the advertised non-integration test suite on main in an isolated environment with the declared dependencies; I did not run credentialed agents or reproduce the benchmark experiments.

Autogenesis is a real and substantial self-evolving agent framework, but the repository is not a complete reproducibility artifact for the paper and its current main is not a healthy release checkout. The paper's concepts are most legible in the v1.0.0 tag published alongside paper v5. Since then, main has become a materially different architecture: it strengthens extension lifecycle and runtime infrastructure while replacing the paper's named optimization layer and explicit planning bus. Any assessment that treats the paper, release tag, and current branch as one implementation will blur important evidence.

The paper separates resources from evolution

The paper's central decomposition has two layers. The Resource Substrate Protocol Layer (RSPL) turns prompts, agents, tools, environments, and memory into registered resources with common lifecycle, state, and version interfaces. The Self-Evolution Protocol Layer (SEPL) applies a Reflect → Select → Improve → Evaluate → Commit loop over those resources, with tracing and rollback. The paper reports Reflection, TextGrad, GRPO, and Reinforce++ strategies and evaluates variants on GPQA, AIME 2024/2025, GAIA, Humanity's Last Exam, and a recent-LeetCode set (paper v5).

The useful architectural move is not any one optimizer. It is the separation between an evolvable artifact's uniform control surface and the strategy that proposes a replacement. In v1.0.0, prompts and tools expose trainable variables, the optimizer package contains the four named strategy families, and a version manager records histories for tool, environment, agent, prompt, memory, benchmark, and skill resources (reflection optimizer, version manager). This gives learning an addressable symbolic update space: selected pieces of prompt text or code can be named, revised, evaluated, retained, and inspected without weight updates.

The paper's Agent Bus supplies the execution loop around that update substrate. A planning agent writes plan.md, assigns planned steps to rounds, derives same-round dispatches for concurrent execution, records results, and asks the model to verify or replan; the bus owns the multi-round loop (planning agent, bus example). This is an explicit planner/scheduler decomposition: natural-language planning selects work, while code commits the plan to a round structure and dispatches it.

Current main is a different system

The rewritten branch keeps the resource-manager idea but changes both the resource space and the orchestration model. Its extension manager recognizes eight modules—tool, agent, prompt, skill, environment, connector, workflow, and memory—rather than merely implementing the paper's five-part presentation. Active artifacts occupy a flat extension tree; every registered version is archived under .versions; a manifest records the active set; manifest replacement is atomic; and rollback copies an archived version back into the active location (extension manager). Generator, optimizer, and evaluator agents are specialized by resource type in configuration, but the paper's src/optimizer implementations of Reflection, TextGrad, GRPO, and Reinforce++ are no longer present.

The orchestrator is now deliberately just another event-driven agent. MetaAgent includes registered subagents in its capability roster, dispatches a model-selected round concurrently through the ordinary agent loop, and handles escalation by replying over a suspend/resume protocol. Its own source says there is no bespoke plan tracking or reviewer gate in code; decomposition and review are prompt-guided (MetaAgent). Typed runtime protocols add escalation, delegation, progress, and control messages, but they do not restore the paper's plan.md scheduler. This branch has converged toward a uniform capability loop, whereas the paper implemented a distinct bus and planning-agent control plane.

That simplification trades codified coordination for flexibility. Making a subagent look like a tool gives every agent the same composition surface and permits concurrent calls, but task coverage, dependency ordering, reconciliation, and final review remain model decisions. A repeated-action guard stops one common no-progress pattern; it does not supply the stronger coordination guarantees of an explicit plan state or hard reviewer gate.

Evolution is versioned more strongly than it is judged

Current main has meaningful mechanisms for mutation safety. An enable_evolving flag blocks overwriting frozen resources; a newly registered component is archived before its manifest entry is committed; failed manifest work unloads the half-committed component; and a default-on smoke gate runs a synthetic task through the real agent loop, rolling an evolved component back or unloading a new one on failure (extension manager, smoke gate). An evolution journal records hypotheses, predicted fail→pass flips, actual gating outcomes, attribution, and reverted hypotheses that should not be proposed again (journal). These are durable control structures, not merely promises in a prompt.

The weak point is selection validity. The smoke gate establishes that a synthetic run completes without a crash, timeout, or constraint stop; it does not establish that the change improves its target task, preserves unrelated behavior, or satisfies a semantic reviewer. The journal can record evaluated outcomes, but it does not itself produce trustworthy evaluations. The top-level orchestration config supplies generator/optimizer/evaluator triads and a reviewer agent, while the runtime explicitly leaves reviewer use soft. Autogenesis therefore has a stronger commit-and-rollback substrate than evidence policy: it can retain and reverse a proposed change reliably without guaranteeing that the proposal deserved commitment.

Authority is bounded by policy, then widened by deployment

The current permission layer assigns entities read_only, workspace_write, or danger_full_access modes and classifies shell, file, destructive, package-management, and administrative operations (permission types). Evaluator agents are configured read-only, which is a useful privilege separation from the writers they judge. The default Bash tool for trusted direct runs is nevertheless danger_full_access (Bash config).

The recommended container path prevents an accidental fallback to host execution, but it is an authority envelope rather than a strong security sandbox. It bind-mounts the live repository, mounts the host Docker socket, uses host networking, and lets the agent spawn sibling containers (sandbox launcher). Those facilities are appropriate for an explicitly trusted build agent; they also mean compromise of the agent container can reach the project and effectively control the host Docker daemon. The safety case depends on deployment trust and evaluator separation, not containerization alone.

Completeness and reproducibility

The repository fails three different completeness tests:

  • Paper implementation coverage is partial but substantive. The v1.0.0 tree contains the named optimizer families, version manager, resource managers, planning agent, bus, benchmark adapters, and experiment scripts. Its own README warns that the codebase is under active refactoring, identifies only examples/run_tool_calling_agent.py as functional, and says other agents are being progressively stabilized (v1.0.0 README). The tag is evidence of implementation, not a claim of a stable end-to-end release.
  • A normal clone is structurally incomplete. Both the tag and current branch record six gitlinks—datasets/hle, datasets/leetcode, libs/AgentOS, libs/EvoAgentX, libs/browser-use, and libs/textgrad—but contain no .gitmodules. git submodule status consequently fails because there is no mapping for the paths. The clone also contains no committed raw benchmark-result bundle that reproduces the paper's reported tables. In particular, the paper's recent-LeetCode data is one of the unresolved gitlinks.
  • Current main is not test-clean. The README says pytest -q is a fast non-integration suite requiring no credentials or peers and the install script runs it as post-install verification (README, installer). With the declared dependencies installed, collection stops with seven errors because autogenesis/tool/__init__.py imports a nonexistent autogenesis.tool.other package (tool initializer). No test body runs. The installer also pipes test output for display without feeding the pytest exit status into its FAILED flag, so its success message is not reliable evidence that tests passed.

The defensible verdict is therefore: the public code is sufficient to study the architecture and locate implementations of most paper mechanisms, but insufficient to reproduce the paper from a fresh clone; v1.0.0 is explicitly transitional, and the latest branch currently fails its own advertised baseline.

Reading against first-principles design-space analysis

Autogenesis supports a broader reading than “an agent learns its prompt.” Its strongest idea is that symbolic system-definition artifacts can share a lifecycle without sharing an implementation or optimizer. That is a deploy-time learning substrate: behavior-changing code and natural-language artifacts become addressable variables with lineage, evaluation hooks, and rollback.

But its resource taxonomy also demonstrates why classification should be treated as a proposed design decomposition, not as first principles. The paper folds MCP and skills into Tool and foregrounds five resource types. Current code separately names skills, connectors, and workflows because the rewrite found operational differences worth separate managers and artifact contracts. The later implementation did not merely add instances within the original five cells; it revised the axes and boundaries by which the cells are formed.

The same issue appears inside the evolution loop. Per-resource generator/optimizer/evaluator triads explore implementations behind the selected resource seams. They cannot by themselves discover that a required correction belongs in a new resource type, a different scheduler boundary, a harder review gate, or a narrower privilege topology. Autogenesis expands the update space substantially, but it does not eliminate the decomposition problem—it relocates the fixed layer to the registries, mutation interface, orchestration loop, evaluation protocol, and authority boundary. First-principles exploration of rival decompositions must precede treating that layer as the space within which self-evolution is allowed to search.

What to watch

  • Whether the authors repair the gitlink metadata and publish the exact datasets, configurations, seeds, traces, and result artifacts behind the paper tables.
  • Whether a stable release pins the paper architecture or explicitly versions the rewritten architecture as a different experimental object.
  • Whether semantic, regression, and task-specific evaluation become hard commit gates rather than prompt-guided orchestration around a crash-only smoke test.
  • Whether resource-specific evolution can propose changes to resource boundaries and orchestration itself, or remains search within the current taxonomy.
  • Whether default deployment separates mutation, evaluation, secrets, and Docker authority strongly enough for untrusted tasks.

Relevant Notes: