Incremental self-improvement synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
A policy that can change how later policies are computed
The strongest supported contribution is a documented route from experience to retained changes in the agent's own instruction-selection policy, together with author-reported benefit when self-modification is enabled. The report compares enabled and disabled IncP/DecP capabilities: cumulative payoff is about 18.7 million versus 1.41 million in a variable-writing task, and 957 million versus 79 million in a maze task. These are attributed comparisons over the described system lives. They do not isolate any particular update, introspection primitive or meta-learning mechanism as the cause of benefit (RT-CLM-experimental-comparison). The unresolved question is whether inspectable execution and controlled comparisons would establish the reported benefit and its mechanism at that same comparison level.
The evidence is Jürgen Schmidhuber's On Learning How to Learn Learning Strategies, Technical Report FKI-198-94, revised January 31, 1995, frozen as the text capture fki198-94-revised-1995-01-31-captured-2026-10-14-text. SRC-1 supplies design and reported operation, not executable implementation, raw traces or an independently inspectable causal experiment. The applicability cutoff is October 14, 2026; it supplies no evidence of later development. The boundary is complete artifact, partial loop: the complete captured report was inspected, but shipped execution coverage cannot be established (boundary, Source register).
This is a classical reinforcement-learning enclosing runtime, not an LLM host. It owns instruction selection, execution, persistent work cells and credit assignment. A mutable table assigns probabilities to integer primitives and arguments at each program address. Entries have localized identities and fixed sampling rules; numerical policy parameters here are symbolic state, not an evidenced distributed-parametric model. The claimed work is lifelong maximization of externally specified cumulative payoff, including changing the probabilities of instructions that compute further policy changes (RT-CMP-probability-distributions, RT-RTE-self-modification).
From sampled instructions to live changes
At birth, storage is zeroed and distributions have maximum entropy. Each run starts at ProgramStart. The interpreter samples a primitive and its arguments, writes them into program cells, checks syntax and executes valid instructions. Stop or a syntax failure halts the run; the lifetime loop starts another without resetting internal storage or the environment merely because of that halt. Death ends the life with cumulative payoff and final state, rather than a per-request answer (RT-RTE-instruction-cycle).
Work cells survive run halts. Program-cell contents are different: the base design samples anew at every visit, so previously successful code is not automatically replayed. The lasting learning product is the probability table, not an archive of successful programs. Addressed work reads and task Read operations retrieve selected values; the interpreter automatically supplies the current address's distribution to subsequent sampling. There is no documented language-model context window or summary-compaction mechanism. The relevant budgets are cells, integer ranges, stack capacity and lifetime computation (RT-OBJ-work-area, RT-OBJ-program-area, RT-BAP-probability-selection, MEM-RTE-variable-readback).
An effective IncP or DecP saves an old distribution and time/payoff bookkeeping, then changes the live table. Fixed operand constraints and a positive minimum probability can veto invalid changes. They do not judge task usefulness before installation. A self-modification program starts with its first effective change and ends through EndSelfMod. Until then, the fixed top-level evaluator is suspended while changed probabilities already govern later instructions. The system computes proposals and ordinary admission without per-change human approval; the programmer supplies primitives, reward and initial parameters (RT-RTE-self-modification, RT-OBJ-stack).
Retention checks historical payoff, not causal explanations
Outside an unfinished self-modification program, completed instruction cycles trigger the unmodifiable top-level evaluator. It compares the current reward rate since the newest surviving program began with the rate since its predecessor began, or birth if there is no predecessor. Non-exceedance causes reverse restoration of saved distributions. Restoration consumes measured time and can invalidate older changes too. Saved distributions support recovery; saved times and payoff totals support renewed comparisons. Neither is a required explanation of why a change was proposed or why it worked (RT-CMP-top-level, EPI-RTE-rate-comparison, RT-RTE-credit-assignment).
The source's recursive usefulness invariant is therefore conditional and historical. Its owner is the fixed evaluator, its enforcement point is the enabled comparison/restoration loop, and its required contracts include accurate reward/time bookkeeping and recoverable old distributions. It neither proves causation nor guarantees future improvement. Unfinished programs can postpone checking without a hard deadline. Rejection reverses direct probability changes only: work-cell writes and environmental effects survive. Proposed teacher protection would bypass selected rejections, and proposed circular history overwrite would reduce recoverable history; neither is established as active in the reported experiments (RT-CLM-usefulness-invariant).
Task observations enter designated cells without a separate truth-admission check. Their warrant depends on encoding and source reliability; the corrupted-input maze variant rules out unconditional observation trust. The evaluator's rate ordering is a conditional numerical derivation from supplied measurements. Its result licenses continued operational use of policy changes within this life, not acceptance of an explanatory or predictive claim. The author's assumption that apparent usefulness persists into the future remains explicitly defeasible (EPI-RTE-input-supply, EPI-OBJ-rate-comparison, EPI-CLM-future-usefulness).
A concrete result boundary
The variable-writing workflow connects these mechanisms to a defined external result. A programmer supplies 30 initially zero variables. Sampled arithmetic and addressing can prepare values; valid Write actions immediately change external variables. Every 1,000 measured steps, the task driver counts variables whose values equal their addresses, delivers that count as payoff/input and resets the external array. Internal work cells persist. Invalid Read or Write halts the current run, not the whole life. There is no delegated worker or approval of each delivered write (WF-RTE-variable-cycle).
The author reports a first event with all 30 variables correct after 286,918 payoff events. This is one maximal score, not stable perfect delivery or terminal acceptance. The workflow specification is source-supported, but no sampled solution trace is supplied. Its payoff is an exact task-defined outcome; it does not identify which retained modification caused the result. Time is a computational-cost proxy charging operations over whole distributions, not measured wall-clock time for every instruction (WF-RTE-variable-cycle, RT-RTE-instruction-cycle).
Independent property assessments
Learning: the enabled/disabled comparisons support reported improvement over the described lives, not independently verified causal benefit or general superiority. Historical payoff measurements also feed automatic retention/restoration of behavior-shaping state consumed by later sampling. That trace-fed write route is documented even though improved capacity is not independently established (RT-CLM-experimental-comparison, RT-RTE-credit-assignment, RT-BAP-probability-selection).
Reflection: GetP supplies a scaled current-policy value to a work cell. Changed policy can change later snapshots, and snapshot-mediated computations can affect later policy. This supports a claimed design affordance for bidirectional introspection; exercised reflection and its contribution to improvement are uninspected. GetP does not expose the evaluator's full saved history, and old copies need not refresh when policy changes (MEM-RTE-getp-read, EPI-OBJ-getp-values, RT-OBJ-stack).
Autonomy: computational proposal, primitive admission and retention are claimed features of the base design. Human primitive/reward design remains outside those per-change roles. This is not evidence of reliable execution or autonomous theory building (RT-RTE-self-modification, RT-RTE-credit-assignment).
Theory-builder conditions 1–4: a formulated-theory route is not established. Condition 1, localized theory content, is uninspected in arbitrary program/work-cell meanings; condition 2, decisions depending on what such a theory says, is uninspected; condition 3, content-directed criticism, is uninspected and is not established by scores selecting policy variants; condition 4, retained criticism shaping another round, is likewise uninspected. Policy persistence does not substitute for criticism persistence. No improvement through criticism of a consumed theory, reflective theory builder or autonomous theory builder is established. These are evidence limits, not claims that the universal instruction language cannot express theories (RT-RTE-self-modification, epistemic bounded conclusion).
Self-improvement: within the probabilistic runtime, over one life, the independent objective is cumulative task payoff. By design, objective-bearing measurements determine retention, the retained result changes its own instruction policy, and that policy reaches live sampling. The fourth link—subsequent operation exercising and causally depending on the change—is attributed by the experiments, not independently established. The supported conclusion is documented improvement-directed self-change and reported exercise/benefit, not independently verified successful self-improvement (RT-RTE-self-modification, RT-BAP-probability-selection, RT-CLM-experimental-comparison).
For these toy-task scenarios, the account supports a policy-change and revisable-retention mechanism, not safe rollback, accepted explanatory knowledge or future reliability. Frozen executable dispatch, evaluator code and linked action/update/reward traces would change implementation and exercise assessments. Independently inspectable enabled/disabled comparisons would strengthen benefit evidence; independent variation of GetP or specific updates would be needed for finer attribution. A documented content-directed criticism route with retained effects on later decisions would change the theory-building assessment. Full-state recovery or bounded evaluation scheduling would change the rollback and delay assessment; neither follows from the current stack mechanism.
Limitations
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| Documentary evidence, not inspected execution | SRC-1; RT-RTE-instruction-cycle, RT-RTE-credit-assignment, boundary | Complete revised report capture, especially §§3–5; no code or raw run artifacts supplied | Implementation fidelity, complete shipped-path coverage, independently observed operation, restart durability or present-day integration | Frozen implementation and drivers, linked execution artifacts, and persistence/restart contracts |
| Ambiguous Read destination and damaged extraction | SRC-1; MEM-RTE-variable-readback, RT-RTE-instruction-cycle, WF-RTE-variable-cycle; report-verification limits | §5.1 prose and Table 2 disagree; other mathematical text is extraction-damaged | Exact Read destination formula and faithful executable/mathematical reconstruction; transfer capability remains supported | Authoritative corrected operand specification and intact mathematical text or corresponding frozen code |
| Unfinished-program halt recovery and evaluation delay | SRC-1; RT-RTE-self-modification, RT-CLM-usefulness-invariant, RT-RTE-instruction-cycle; report-verification limits | §3.2 and §4 distinguish run halt from EndSelfMod but supply no executable interaction protocol | Verified recovery during an unfinished self-modification program or any hard bound on delayed evaluation | Explicit halt/completion state transitions, enforcement code and traces; a bounded scheduling rule for a hard deadline |
| Historical retention is not causal or future warrant | SRC-1; RT-CLM-usefulness-invariant, EPI-CLM-future-usefulness, RT-CLM-experimental-comparison | §§3–5; enabled/disabled modifier capabilities are bundled, with narrative results only | Particular-update or GetP attribution, general superiority, future benefit, or safety of surviving effects | Inspectable controlled comparisons at the claimed attribution level, independent component variation, and separate evidence over the intended future domain |
| Unresolved representations and exercised memory/reflection routes | SRC-1; RT-OBJ-work-area, MEM-RTE-getp-read, MEM-RTE-variable-readback, EPI-RTE-input-supply, RT-OBJ-stack; profile verification | Documented in-life memory, §§3–5; arbitrary work histories and underlying navigation representation remain unresolved | Complete memory lineage/consumer inventories, beneficial exercised introspection or Write-to-Read use, machine-produced accepted explanatory knowledge, or a system-complete negative about theory building | Decoded retained parts and provenance, actual consumer traces, and any formulated claim/criticism/acceptance route with its later effects |
The supplied reconciliation identifies no unresolved cross-report conflict. Report verification contributes the two specific Read and halt-handling limits carried above; profile verification declares no additional limits. No direct source reads were needed for this synthesis.