RSIAgent synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From task execution to an independent local judgment
RSIAgent's strongest supported contribution is an implemented loop that turns task experience and terminal verification evidence into a revised bank for future agents. It also uses outcome-fed Curriculum decisions to select practice before a fresh target attempt. These are substantive improvement-directed mechanisms, not just a storage API. Whether a particular update embodies sound criticism, changes later decisions or improves capability remains unestablished (RT-RTE-learning-admission, RT-RTE-curriculum-search).
This account covers the shipped enclosing runtime at Git revision 4d7ef43770dda341fccd9a5743db92407687c615, with an analysis cutoff of 2026-10-14. SRC-1 is the frozen RSIAgent repository. The whole-system functional boundary includes both OSWorld and Agents' Last Exam (ALE) drivers, task execution, learning, memory publication, recovery and later consumers. Findings are code-grounded implementation and attributed design; README outcomes are reported operation only. No target execution, tests, model calls, inspectable run traces or causal experiments were supplied or performed. External benchmark implementations, provider internals and live infrastructure are outside the inspected source evidence (boundary and evidence, Source register). This synthesis uses the accepted members and makes no additional source reads.
The operator starts a benchmark driver. Python controls phases, role construction, action parsing and lifecycle gates; the Actor model chooses actions and generated Python/Bash programs. Programs run in the selected VM, Look returns pixels or delegated vision text, and Ask reaches a supplied simulator callback rather than a human approval gate. Actor Done proposes a candidate, not correctness. A separate Verifier context investigates the task state and can return repair findings or a terminal decision. An official benchmark score comes later, after harness termination, rather than serving as a gold answer inside the inspected learning loop (RT-RTE-task-execution, RT-RTE-candidate-verification, EPI-RTE-official-score).
The Verifier is instructed to require affirmative evidence for every material requirement. Its report token has immediate control authority, but the full-Agent parser checks unambiguous decision transport, not evidential adequacy. Legacy Done probes likewise use lexical screening and PASS/FAIL output rules, not a semantic proof. Thus local acceptance has a task-specific intended criterion without an inspected instance showing that an evidence-consuming decision met it. Observations themselves arrive with instrument/source warrant unknown; neither a delivered observation nor a well-formed PASS certifies its interpretation (EPI-RTE-observation-import, EPI-RTE-probe-check, EPI-RTE-candidate-check, EPI-RTE-verdict-admission).
Effect protections are also path-specific. Effect-isolated Verifier Programs use namespace, mount and privilege controls and fail closed when their selected primitive is unavailable. The public OSWorld baseline instead selects rollback mirror, restoring checkpoint-covered guest state. Remote-service effects are outside that transaction. Trusted image operations and legacy probes have different boundaries; Actor-generated code does not inherit Verifier confinement (RT-RTE-candidate-verification).
From terminal evidence to retained advice and new practice
After a grounded terminal PASS or FAIL, phase policy can resume the same Actor with its active history and final Verifier report. The Actor distills lessons, reconciles the whole bank and writes a separate diagnosis for Curriculum. Instructions explicitly distinguish what was verified from why it happened and request qualification or removal of unsupported explanations. The Actor owns content and semantic readiness. Neither Verifier nor Curriculum grades the memory, and no per-lesson human approval is wired (EPI-RTE-memory-proposal, MEM-OBJ-learning-diagnosis, RT-RTE-learning-admission).
Host admission is a different decision. Transport, path and target-corpus overlap checks can veto publication; the terminal promotion caller rejects the entire candidate if the audited tree differs. Successful admission journals and installs the complete tree. Parallel branches receive the current canonical bank before their serialized reconciliation, rather than simply merging stale snapshots. The staged installer has rename/exception recovery, not a demonstrated always-visible crash-proof swap. A separate destructive replacement API does not inherit that guarantee (EPI-RTE-memory-admission, EPI-RTE-bank-retention, MEM-OBJ-memory-journal).
This publication gate establishes neither a lesson's truth nor its causal explanation or transfer. No generated bank supplies claim-level evidence of accepted knowledge or post-acceptance integration. Files may contain advice, programs or other bytes; their names do not resolve their form or authority. Whole-tree replacement can withdraw active advice while historical snapshots retain it (MEM-OBJ-memory, EPI-RTE-memory-proposal).
Later Actors receive an uploaded bank and a filename/size inventory, not automatic injection of every file body. The foreground Actor chooses file reads through Programs. Retrieved advice is explicitly fallible model-interpreted context, not an enforced lesson interpreter. Attachment and requested content reads are distinct operations; planned memory assignment does not establish delivery or use. Standalone OSWorld upload failure can continue memory-OFF, while Phase-2 and ALE attachment failures raise (RT-BAP-actor-method, RT-RTE-task-execution, MEM-OBJ-memory).
Curriculum separately retains notes, role history, outcome reports and the Actor diagnosis. It chooses projects and readiness, not correctness or memory approval. A configurable empty-bank ablation removes canonical-bank exposure without removing those other channels. Automatic trace-fed bank and continuation-summary writes have later consumers; that architectural result does not establish improved future capacity. Curriculum note-specific trace dependence is less resolved than its wired retention channel (MEM-OBJ-curriculum-notes, RT-RTE-curriculum-search, MEM-RTE-worklog-fold).
A representative progression to frozen evaluation
The OSWorld workflow connects these mechanisms into a delivery protocol. Phase 1 supplies an exploration bank. Phase 2 starts fresh target environments and roles, learns from grounded outcomes, and delegates practice to persistent Curriculum. A project requires a usable handoff and captured replayable fixtures; malformed publication returns to Curriculum. Practice outcomes and revised memory feed subsequent rounds. READY starts a fresh target attempt; STALLED starts one final fresh test rather than declaring success. PASS may still be learned from and reviewed under curriculum_review; practice after PASS requires another target test (RT-RTE-curriculum-search, WF-RTE-phase-transition).
Protocol completion need not mean target success. A final post-STALLED FAIL can advance, but UNVERIFIED cannot. Infrastructure failure is not converted into a semantic verdict. The pipeline validates local result fields, bank lineage and evaluator-boundary records before freezing the bank. Phase 3 copies it, disables curriculum and host writeback, and checks host hashes before publishing aggregate task records and scores. The frozen host tree does not imply immutable guest copies or model reliance. ALE has a distinct host/worker grading boundary: worker completion alone is not an official score (WF-RTE-phase-transition, RT-RTE-task-execution, EPI-RTE-official-score).
Role continuity does not require complete conversational continuity. Conditional Actor folding substitutes a capped WORK LOG and protected tail for older turns; stalled recovery can discard the conversation while keeping the machine and a selected handoff. Full-history profiles bypass folding. Verifier decoder recovery instead retains substantive action/observation pairs, including exact archived images, while dropping actionless exchanges. Scratch persistence is separate and can fail under explicit recovery tolerance. Selected lineage recovery checks archived hashes, configurations, committed memory and role evidence; it does not create a new correctness judgment (MEM-RTE-worklog-fold, MEM-RTE-verifier-replay, MEM-OBJ-verifier-scratch, RT-RTE-lineage-recovery). Curriculum additionally has project-boundary image eviction, so archived pixels do not imply visual continuity in every later call; the retained inventory is incomplete here (report verification limit).
What this establishes about improvement
For the agent-plus-bank across exploration and later task use, the independently specifiable objective is improved task capability. Experience can feed an update to the system's own retained guidance and a later Actor can consume it. This supports a dispositional improvement-directed self-change mechanism. No trace establishes evidence-responsive determination of a particular update, actual semantic uptake, or later behavior causally depending on that change. Exercised and successful self-improvement therefore remain unestablished, independently of the author's fixed-parameter claim. Completion calls do not train weights, but configured model endpoints are not immutable weight identities (RT-RTE-learning-admission, RT-CLM-training-free, RT-CMP-actor, RT-CMP-verifier, RT-CMP-curriculum, RT-CMP-vision-model).
The runtime assesses theory-builder conditions independently for bank learning and Curriculum search: 1, localized formulated content: afforded through inspectable whole files or notes; 2, decisions depending on that content: claimed in instructions, not observed; 3, content-directed criticism with revision or changed reliance: claimed, without a completed instance; 4, iteration: wired through next-round delivery, without proof that a criticism result shaped the next round. Whole-text replacement is available; finer claim segmentation is not enforced. These findings do not establish satisfaction of all four conditions (RT-RTE-learning-admission, RT-RTE-curriculum-search).
Criticism improving capacity for future action is not demonstrated. Reflection is requested about Actor bottlenecks and outcomes and has return paths through practice and memory, but both representation-mediated causal directions are unobserved. Reflective criticism of the system's own method is not established. Computational role assignment and automatic admission are supported; autonomous performance of every theory-building operation is not demonstrated. These are separate conclusions, not a single negative verdict (RT-RTE-curriculum-search, EPI-RTE-memory-proposal).
For target-conditioned adaptation, the implementation connects diagnosis, practice, retained advice and fresh testing. It does not establish held-out generalization: the public OSWorld batch can use the same task in development and evaluation. README score gains are attributed reports with selected retries and differing budgets, not matched repeated-run component experiments. Deterministic cohort arithmetic is licensed only conditional on its imported scores; it neither authenticates manuscript provenance nor isolates memory or criticism effects (RT-RTE-task-execution, EPI-CLM-reported-scores, EPI-RTE-score-report).
An evidence-linked generated bank, before/after revisions and later consumption traces would change the assessment of semantic criticism and activation. Matched outcome comparisons would address benefit; independently varying memory or criticism would be needed for those component effects. A genuinely separated development/evaluation task cohort would address transfer beyond target-conditioned adaptation. Enforced evidence requirements for verdicts or lessons would change the present transport-versus-warrant boundary, but would still require evidence of operation and benefit.
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No inspectable operation or causal comparison | SRC-1; RT-RTE-learning-admission, RT-RTE-curriculum-search | Frozen shipped control and learning paths, not executed | Actual theory consumption, completed criticism, reflection's causal directions, exercised self-improvement and capability gain | Linked execution traces of update determination, admission and later behavior; suitable outcome and component comparisons |
| Generated payloads and initial provenance unresolved | SRC-1; MEM-OBJ-memory, MEM-OBJ-curriculum-notes, MEM-OBJ-verifier-scratch | Storage, instructions and consumers inspected; no generated bank, notes or scratch supplied | Complete form/authority inventory, universal trace-derived provenance, evidence-linked lesson acceptance and rationale preservation | Actual payloads with authorship/admission records and evidence-to-revision-to-consumer links |
| External implementations and exact provider identities uninspected | SRC-1; RT-CMP-actor, RT-CMP-verifier, RT-CMP-curriculum, RT-CMP-vision-model, EPI-RTE-official-score | Local adapters and endpoint configurations only | Independently fixed weights, provider-side accumulated-state classification, official evaluator correctness and end-to-end reliability | Immutable model identities and authorized provider/evaluator implementation or execution evidence |
| Recovery coverage is partial | SRC-1; RT-RTE-lineage-recovery | Selected Phase-1/Phase-2 admissions and verified-memory continuation; other dispatched reconstruction bodies not fully inspected | Exhaustive reconstruction, admission or recovery guarantees and complete memory inventory | Inspection and records for remaining variants, plus execution evidence for claimed recovery outcomes |
| Curriculum project-boundary image eviction omitted from retained inventory | SRC-1; RT-RTE-curriculum-search, MEM-OBJ-durable-images, MEM-RTE-verifier-replay; report verification limit | Verifier replay/image archive inspected; verifier additionally found attachment removal in explore/practice_loop.py, invoked after waves by explore/phase1_wave.py |
Complete image eviction/reload inventory and universal Curriculum visual continuity; no implication that Verifier replay drops substantive pixels | Dedicated records tracing removed attachments, retained text/reissue-Look note and subsequent image reload paths |
| Verdict and memory gates are not semantic certificates | SRC-1; EPI-RTE-verdict-admission, EPI-RTE-memory-admission | Full-Agent token parser and target-learning corpus gate | Guaranteed evidential adequacy of PASS, true lessons or accepted knowledge from publication alone | Evidence-consuming decisions for named claims and scopes; stronger enforced criteria if a guarantee is sought |
| Context and effect protections have scoped guarantees | SRC-1; MEM-RTE-worklog-fold, RT-RTE-candidate-verification, RT-RTE-task-execution | Conditional summary/restart, protected Verifier execution and benchmark attachment paths | Summary fidelity, system-wide noninterference, remote-effect rollback, guest-bank immutability or universal memory delivery | Fidelity checks and mode-specific effect/transport traces under explicit environmental contracts |
| Outcome reports do not identify transfer or component effects | SRC-1; EPI-CLM-reported-scores, EPI-RTE-score-report, RT-RTE-task-execution | README aggregates, report arithmetic and public target-conditioned OSWorld planning | Reproduced manuscript scores, matched improvement estimates, held-out generalization or isolated memory/critic benefit | Provenance-linked cohort artifacts, matched budgets/retries, distinct held-out tasks and independent component variation |