OpenRSI synthesis

Type: agentic-system-analyses/types/agentic-system-synthesis.md

Bounded synthesis

OpenRSI ships an improvement plane with three separately invoked parts: Gym constructs executable ML task packages, Evo searches for candidate programs, and ERL supplies supervised and reinforcement-learning paths for the proposing operators. Its strongest supported contribution is the connection of execution feedback to later program proposals through retained experiment state, alongside explicit training-input paths for changing operator parameters. These are implementation findings, not observed capacity gains. The release itself describes bounded improver training rather than a solution to general recursive self-improvement: RT-CLM-meta-evolution.

The evidence is SRC-1, the OpenRSI repository at commit 71ae803a035d5e3b78c19fa49ed9f67d0550cbaa, with an applicability cutoff of 2026-10-19. The boundary is complete artifact, partial loop, not whole-system coverage. Inspected code includes shipped drivers, vendored search, retained state, checking and training handoffs. Documentation supplies design intent and attributed reports. No target execution or causal experiment was supplied or performed. External weights, corpora, task assets, deployed services and the separate RL SLIME dependency prevent a complete historical training-to-deployment account. The vendored SFT trainer is not evidence for the external RL trainer's internals; see the Source register.

Construct a task, then establish what its checks mean

Gym's build command defaults to dry-run; execution must be enabled. Once enabled, software routes acquisition and model calls, admits compiled preparation source, executes it, checks expected outputs and retries failed preparation. The model supplies descriptions and source; software owns continuation and admission. Starting the workflow is not per-file human approval. Retry context selects the last two retained preparation messages rather than automatically carrying the entire trace. Generated metric source is compiled and written, not thereby certified semantically correct: RT-RTE-gym-build.

Separately invoked checks offer different assurances. Metric-check executes the task's own metric against its sample submission and reference answers and requires a finite numerical result. This establishes a task-local smoke check, not independent authentication of reference truth or metric fidelity: EPI-RTE-metric-check. Task-quality evaluation can use model judgment, skip it, or hard-gate on deterministic validation failure. Normalized quality scores produce reporting recommendations; a low recommendation alone is not the CLI failure predicate and no automatic downstream training or release veto was established. See EPI-RTE-quality-check and EPI-RTE-quality-recommendation.

Execution authority is broader than the bound perception tools. Generated Python can run through the default process path. Gym's optional isolated worker requests networkless, read-only, non-root containers with explicit writable mounts and resource restrictions, conditional on the selected path and suitable deployment. The build coordinator remains a process worker. These controls do not establish blanket confinement of Gym, Evo's remote sandbox or NatureBench's local/Docker/SCM alternatives: RT-RTE-gym-build, RT-RTE-naturebench-execution.

Propose, execute, repair and deliver a program

A representative supported workflow is generation-mode Evo MLE search. The configured evaluation driver writes task and runner configuration, launches task/sample processes, constructs the task adapter and solver, and restores the journal. Software owns the outer policy: initial drafts, subsequent parent/operator selection, execution, parsing, repair, population admission and stopping. Separately configured draft, improve, debug, crossover and analyzer model clients propose or interpret content; this does not mean one exact model serves every role. Evo inference changes programs and search state, not those operator parameters: RT-RTE-evo-generation, RT-CMP-generic-llm.

Candidate source crosses an executable-action boundary into the task adapter. In MLE, the configured remote sandbox returns status, logs and fitness; its task-supplied metric is an external execution contract, not a universal correctness oracle. Parser validity and metric policy determine debugging and selection. Scoring is not uniformly independent holdout evaluation: supported configurations include analyzer fallback, optional trust in a printed validation score and test-score substitution. These alternatives establish conditional scoring semantics, not an observed cheating event or score error: RT-RTE-evo-generation, RT-BAP-candidate-execution.

A buggy candidate enters bounded debug. Each repair creates a new proposal and passes through the same execution/parser route. Successful repair supplies the last repaired node for generation admission; exhaustion discards the individual from that generation, not necessarily from retained history. Exception handling covers the evaluation/parsing block, not every operator failure. Time checks after unsuccessful attempts are not hard pre-attempt wall-time guarantees, and repair does not roll back external effects: WF-RTE-debug-cycle.

Delivery means exported files and task statistics, not proven external acceptance. The runner reselects from available candidates; if none has a valid selection score it can choose code using a seeded random fallback, and with no candidate it exports an empty string. Thus valid_code_final.py is not a verification certificate. Enabled final submission evaluates selected best_code, not necessarily the transformed text written to submit_code.py. A false final-submit setting makes this runner's final test loop empty while leaving export and earlier search evaluation intact. NatureBench supplies a reachable configurable false case; direct adapter capability does not demonstrate a supported bypass of the runner decision. See RT-RTE-final-submit and its workflow annotation.

Asynchronous search is a distinct scheduling route: workers propose/evaluate concurrently, while separate locks coordinate sampling and commit. Stop conditions can reject completed work. This is not evidence of generation-mode equivalence, strict budgets for all in-flight work, race freedom or speedup: RT-RTE-evo-async.

Carry experience into the next proposal

Evo retains more than final code. Its journal holds programs, plans, raw output, analysis, metrics and ancestry. Cards derive selected experiment metadata; a strategy board aggregates it. Optional rich summaries request method descriptions and parent comparisons from a model. These are distinct objects with different representations and recovery paths: MEM-OBJ-journal, MEM-OBJ-experience-card, MEM-OBJ-strategy-board, MEM-OBJ-rich-summary.

The operative contribution is a wired trace-to-later-consumer chain. Software derives and persists cards, reattaches them on resume, selects parents/ancestors/siblings or same-error history, and supplies chosen fields to a later operator call. Targeted memory is configuration-dependent; base journal memory is an alternative or optional addition. This is mostly host-initiated supply, not a model querying a general memory service. The receiving model sees selected values, not live pointers into the archive. Item caps, character truncation and Gym's message tail are different controls, not a system-wide token-fit guarantee: MEM-RTE-experience-cycle, MEM-RTE-operator-context.

Recorded metrics and flags can directly rank candidates, select operations or exclude them from particular contexts. Summary prose instead has advisory force through model interpretation. Summary admission requires usable nonempty fields, not checked explanatory truth. Reuse from node, card or cache is not semantic revalidation; 420-character display fields can lose qualifications. Arithmetic derives facts under its inputs and category definitions, while heuristic family labels do not establish scientific categories. No actual summary response supports deciding whether it faithfully reshapes evidence, derives a claim or conjectures a new explanation. See EPI-RTE-experience-derive, EPI-RTE-summary-produce, EPI-RTE-summary-retain and EPI-RTE-summary-consume.

Train the operators without conflating retention and acceptance

SFT and RL add durable training paths, but they are not automatic stages of every Evo search. SFT collection and selection can filter scored trajectories, reconstruct evolutionary/debug segments, deduplicate normalized messages and reject oversized examples. The launcher also accepts arbitrary caller-selected datasets, so launch does not attest to all upstream gates or rollout ancestry. The inspected SFT search database stores rejection metadata without requiring rejection-policy acceptance; checkpoint reconstruction can omit prior feedback. Training consumes message-derived tokens and loss masks, not a fresh execution check of every example: RT-RTE-sft-training, MEM-RTE-sft-reuse.

RL wires generation, a model hack classification, sandbox eligibility, reward arithmetic and program insertion to an external training environment. HTTP 200 or explicit invalid categories can permit storage. Later configured selectors choose parents and experience; some invalid retention is diagnostic, not endorsement. Fitness-based pruning is separate from insertion, and evo and airaevo select different implementations. Local retained-context updates and their later prompt consumers are supported independently of the external numerical optimizer: RT-RTE-rl-training, MEM-OBJ-rl-programs, MEM-RTE-rl-reuse.

Automatic trace-fed retained updates with later consumers are therefore supported architecturally in Evo, the rollout-derived SFT branch and local RL experience reuse. That contribution does not require a new explanatory claim or measured benefit. It also does not resolve the full memory inventory: profile verification accepts only partial coverage across all ten comparison axes. Textual guidance, executable source and numerical training inputs have different consumers and force; MEM-BAP-experience-guidance and RT-BAP-training-parameters keep them separate.

What the improvement evidence supports

For bounded executable tasks, the implementation supports iterative proposal, task-relative selection, retained feedback and training-input preparation. It does not demonstrate reliable repair, trustworthy task construction, measured future capacity gains or an exercised training-to-Evo deployment chain. The reported benchmark contrasts are model–harness results; the Max setting bundles priors and asynchronous search and is not a pure model treatment. Without raw outcomes and independently varied components, attribution cannot isolate training, experience or scheduling: EPI-CLM-model-harness. The claimed restore, optimizer steps and serving synchronization remain author-reported operation, not an inspected run: RT-CLM-training-validation.

The records support improvement-directed self-change pathways through retained behavior-shaping context and operator-training inputs, distinct from improvement of external candidate programs. Within the inspected boundary and later-round/checkpoint horizon, however, subsequent operation causally dependent on those changes is unobserved. Exercised self-improvement and successful improvement remain unestablished. Improved capacity through criticism of a consumed theory is likewise not established. The runtime declares no theory-builder conditions 1–4 assessment; this synthesis infers none from plans, scores or retained summaries. Reflection and autonomous theory building also remain unestablished independently, not disproved by missing traces. The narrower computational ownership of the selected search workflow is supported. No inspected route establishes an accepted ampliative explanation and its subsequent lifecycle integration; unassessed skills, metadata and variants prevent a system-complete negative. See RT-BAP-training-parameters and the epistemic conclusion.

A linked trace from objective feedback through retained update to later consumption and causally dependent behavior would change the exercised-path assessment. Exact corpus/checkpoint identities and endpoint mapping would strengthen training lineage. Controlled comparisons with independently varied training, experience and scheduling would permit finer benefit attribution. Actual summaries paired with their source evidence and a consequential semantic acceptance decision would change the explanatory-knowledge assessment. Recovery round trips preserving evaluator metadata would strengthen restart claims. These are missing evidence or concrete system changes, not assumptions needed to retain the positive implementation findings.

Limitations

limitation	affected SRC IDs and record citations	inspected boundary	conclusion prevented	evidence that would resolve it
Evaluator-metadata recovery mismatch; separate recovery surfaces do not prove semantic equivalence.	SRC-1; MEM-OBJ-journal, MEM-RTE-experience-cycle, MEM-OBJ-rich-summary; report limit.	Journal exporter stores nested metric metadata; reconstruction reads flattened keys. Cards and summary caches have separate loaders.	Complete evaluator-metadata recovery, crash-atomic cross-file state and semantically equivalent restart. Code and metric-value recovery remain supported.	Corrected serialization contract and linked save/restore artifacts comparing metadata, cards, caches and later selection.
Partial inventories retain available but uninspected alternatives.	SRC-1; MEM-OBJ-task-skills, MEM-RTE-sft-reuse, MEM-RTE-rl-reuse, epistemic coverage; report limit.	Evo-local database callers, skill contents/provenance and task-key matching, full SFT variants, RL helper/cache/lifecycle consumers; broader metadata and evaluator coverage remains partial.	Cross-implementation database equivalence, full maintenance/restart inventories, universal memory semantics or a system-complete negative about epistemic acceptance or criticism.	Source-grounded producer/consumer and maintenance traces for these alternatives, including final payloads and admission conditions.
Partial storage comparison omits supported local classifications.	SRC-1; RT-RTE-gym-build, RT-CMP-rl-model; profile limit.	Gym persists generated source/package material; RL has distributed actor/serving parameters, but the comparison groups these storage units as uninspected.	Object-level storage completeness or an inference that these units contain no classifiable retained state. Existing files/model-weights existence findings remain valid.	Per-object classification retaining Gym files and RL parameters separately from unresolved lifecycle and persistence organization.
External assets and execution evidence are missing.	SRC-1; boundary, RT-CMP-sft-model, RT-CMP-rl-model, RT-CLM-training-validation, EPI-CLM-model-harness.	Frozen repository, external RL optimizer, unpinned supplied/deployed weights, corpora and services; documentation reports only.	Exact historical lineage, independent reproduction, deployed isolation, observed parameter updates and finer causal improvement attribution.	Registered execution artifacts, immutable corpus/model/deployment identities and appropriately controlled component comparisons.
Scoring and execution assurances are conditional on route and configuration.	SRC-1; RT-RTE-gym-build, RT-RTE-evo-generation, RT-RTE-naturebench-execution, RT-RTE-final-submit.	Optional isolated workers, default process/local alternatives, external task metrics, analyzer/self-score/test-score policies and runner-local final-test control.	Blanket confinement, universal independent holdout scoring or interpreting export as successful final testing. Direct adapter capability does not establish a supported bypass.	Path-specific deployment and scoring traces, validated oracle inputs and explicit end-to-end controls over each asserted guarantee.
Summary availability does not establish explanatory warrant or beneficial activation.	SRC-1; EPI-RTE-summary-produce, EPI-RTE-summary-retain, EPI-RTE-summary-consume, MEM-BAP-experience-guidance.	Shape-based summary admission, node/card/cache reuse and selected/truncated advisory delivery; no actual output/consumer trace.	Faithful compression, novel or accepted explanatory knowledge, theory-directed learning, exercised self-improvement or measured benefit from this pathway.	Input/output semantic comparisons, evidence-consuming acceptance where claimed, and later behavior traces with causal comparisons for benefit.

No unresolved report conflict was identified by report verification. Source-read log: none; this synthesis uses the supplied accepted members only.