A proposal-selection improvement loop requires search, evaluation, and operative retention

Type: kb/types/note.md · Tags: foundations, computational-model, self-improving-systems

A proposal-selection improvement loop is the architecture of improvement in which candidate changes are generated, evaluated with a possibility of non-adoption, and selectively made operative. It is a named subtype, not the whole of the phenomenon: a self-improving system needs its changes to be responsive to evidence bearing on an improvement objective, and evidence may instead determine an update directly — gradient-, reward-, error-, or viability-driven — with no candidate ever standing to be rejected. What follows is the anatomy of the subtype, and it applies with full force exactly there.

A proposal-selection loop requires three functions: search brings a candidate change into consideration, evaluation supplies grounds for accepting or rejecting it, and operative retention preserves an accepted change with behavioral authority. Remove any one and the loop does not close — a change nobody proposed, nobody could reject, or nobody will ever act on.

The loop is therefore narrower than self-modification. A blind, accidental, or unconditional rewrite may change later behavior without applying any criterion; a transient rewrite may fail to preserve the result. Both can count as self-modification, but neither closes a proposal-selection loop. Conversely, the three functions can close the loop in a system that is not reflective at all.

A terminology note: the concept descends from Ashby's adaptation — his ultrastable system, examined below, is the conceptual ancestor even though it classifies outside the subtype — but it is named for what the loop aims at rather than by his word for it. Everyday adaptation is transient compensation, an eye adjusting to the dark, and retains nothing; retention is one of the three requirements. Where this note says adaptation or adaptive, it means Ashby's phenomenon; the loop is the proposal-selection improvement loop, and improvement loop below abbreviates it.

A reflective system supplies one possible causal path into this loop. Through intercession — an operation that changes the system through its causally connected self-representation — it can modify a represented aspect of itself. Making that path available does not itself provide search, evaluation, or retention.

Search determines what enters consideration

Search brings an unrealized change under consideration. It may include:

  • detecting a problem, opportunity, or adaptation signal;
  • selecting the aspect and operation to change;
  • generating one or more candidates;
  • allocating effort and deciding when to stop or escalate.

At minimum, search must produce a candidate from a space in which other possible changes remain unrealized. It need not compare several candidates at once or operate autonomously. A maintainer may choose the problem, a model may draft a candidate, and a script may enumerate alternatives within one declared socio-technical loop. Assigning those functions establishes the loop's boundary; it does not make the loop reflective.

Search reach and evaluation strength are independent limits:

Evaluation cannot select a candidate that search never reaches.

A strong verifier can improve judgments within a narrow generator's reach, but it cannot expand that reach. Automating KB learning is an open problem gives one concrete search space—extract, split, synthesize, relink, regroup, reformulate, retire—whose judgment-heavy parts remain substantially human-driven.

Evaluation determines which changes may remain operative

Evaluation applies criteria to a proposed or already actualized change. Its result must be able to affect selection, rollback, or continued retention. Evaluation is non-vacuous only if some possible result permits rejection: an unconditional trigger is not an evaluator merely because it precedes a transition, and a conditional trigger whose only effect is to launch the next variation is not one either. The verdict must control an operation distinct from producing the next candidate — select, discard, block, roll back — so that rejecting a change and merely changing again are different events in the mechanism.

Oracle is shorthand for the component or procedure that supplies the evidence or judgment. It may be a proof system, test, validator, empirical measurement, rubric, model evaluator, human review, or some combination. The oracle-strength spectrum grades these mechanisms, while the boundary of automation is the boundary of verification explains why constructing an adequate oracle is often harder than generating candidates.

Any judgment remains scoped to what the check establishes. An oracle may accept a candidate under specified criteria without establishing that the change is globally beneficial. Search and evaluation may be performed by the same person or process, but they fail in different ways and improve by different means. They are analytically separable rather than independent: automating one changes the load on the other.

Operative retention makes the change consequential

Acceptance alone does not make a change consequential. Operative retention combines persistence with an authority path through which the retained result can affect later behavior. In behavioral authority terms, the change needs a consumer, a channel, and a force.

  • A reviewed note that no future reader or prompt-assembly step loads has no consumer.
  • An approved patch that is never merged has no channel.
  • A generated validator that no command invokes has no force.

In each case, search ran and evaluation passed, but the improvement loop remained open: the artifact exists without becoming behaviorally consequential.

A behavior-determining objective chains a second loop instance

Behavioral authority's consumer/channel/force vocabulary splits retained artifacts into two force families: a knowledge artifact, consumed as evidence, reference, or advice, and a system-definition artifact, consumed with instruction, enforcement, routing, validation, or configuration force. The three functions above are stated per loop instance, and one instance can close entirely within the knowledge-artifact family: accepting and integrating an ampliative conjecture — the closing phases of a discovery-lifecycle instance — retains it as a connected claim about the target system, which is genuine operative retention when the improvement objective is explanatory.

An objective that instead targets the system's own behavior-determining organization is not reached by that retention, because a knowledge artifact carries no instruction, enforcement, or configuration force by itself. Reaching the system-definition layer takes a second, chained loop instance, not a different kind of retention step: its search recognizes the first loop's retained knowledge artifact as a promotion candidate, its evaluation applies actionable methodology's four elements — mapping, operator, available operations, reachable target — alongside the correctness and coverage bets that decide whether promoting is worthwhile, and its retention gives the result instruction, enforcement, or configuration force under a maintenance regime of its own. Methodology with incomplete coverage and its live theory fallback form a two-layer execution system develops that second instance and names its trigger promotion. A candidate that never gets chained into that second instance is the same open loop the three examples above already show, now with a name for which instance stalled.

Feedback is what makes later iterations responsive

The loop can repeat without feedback — on a timer, on a fresh request, on an unrelated proposal. What it cannot do without feedback is respond to what the last iteration did: the consequences of retained changes have to reach later search or evaluation, through tests, usage traces, incidents, human observation, or revised criteria. Feedback is a condition on learning from the loop, not on running it.

Retained design rationale is one such channel. Recorded constraints and rejected alternatives can narrow later search, while prior evidence can explain why an evaluator once accepted a choice. The record neither establishes that the same oracle remains adequate after the environment, criteria, or risk posture changes nor affects the loop unless a later consumer reads it. Design rationale management in Commonplace documents this useful but non-mechanical retention path.

Boundary cases clarify the claim

Cybernetician W. Ross Ashby's ultrastable system marks the subtype's edge from just outside it, and its exclusion follows from the evaluation criterion above, not from a missing component. The electromechanical Homeostat has exactly one evidence-responsive transition: when essential variables leave viable bounds, the parameters jump to new random values (Ashby 1960, chapters 7–8). That single jump both discards the incumbent configuration and produces its successor — rejection is not an operation distinct from generation — and a configuration that restores viability persists through equilibrium, with nothing whose function is to accept it. The functions collapse into one trigger, so under the definitions here the machine is a non-reflective, direct viability-driven self-improving system, not an instance of this subtype.

What the Homeostat does admit is a functional variation–selection–retention reading: configurations vary, viability determines whether variation continues, and the survivor persists through non-displacement. That reading is an analyst's reconstruction, not architecture, and its value is to mark the floor of each function — search as a draw from a random-number table bearing no relation to the problem, evaluation as a one-bit viability boundary that ranks nothing, retention as equilibrium, a configuration surviving because nothing is left to displace it. Read this way, the Homeostat is the cheapest demonstration of what a stronger generator and a real oracle actually buy. Reflection is still not a premise of the decomposition, but the case that shows it is a genuine instance: an evolutionary strategy runs an explicit generate-and-select loop over parameters nothing inside it can read.

The Homeostat's contrast with a gated system is architectural, not a difference of gate strength: a gradient learner has no evaluator either — online gradient descent adopts every step the revealed cost dictates, with no accept/reject anywhere (Zinkevich 2003) — and the Homeostat stands with it, on the excluded side of the boundary just drawn. Gödel machines sit inside the subtype at its formal extreme, a proof-mediated gate rather than none at all; that architecture is developed in their own note.

Reflection is a separate axis from this exclusion: the Homeostat is also non-reflective, and what that costs is addressability, not category membership — evidence-responsive operative change to the system's own organization, with or without a self-representation and with or without a gate, is what makes a self-improving system.

What the decomposition claims

The three functions are analytically separable, not architecturally separate. One process may perform several of them — a maintainer who notices a problem, drafts the fix, and merges it performs all three — and evaluation may run before a candidate becomes operative or after. Co-location has a floor, though: the functions must remain causally distinguishable even when one process performs them — rejection, in particular, must be an event distinct from the arrival of the next candidate. Where they collapse into a single evidence-triggered transition, as in the Homeostat, the loop is not weakly present; it is absent, and the pathway is direct. The decomposition specifies what the loop must accomplish, not a sequence, a component diagram, or a division of labour. Its use is diagnostic: when a loop stalls, ask which of the three is missing rather than which component failed.

The status claimed here matches how the neighboring self-adaptive-systems field treats its own loop models: MAPE-K and its relatives are presented as reference models for engineering adaptation, not as the definition of it (Weyns, Software Engineering of Self-Adaptive Systems), and a systematic review of that literature finds no settled formal definition from which any single loop architecture would follow (Petrovska, Erjiage, and Kugele 2025). The proposal-selection decomposition is offered in the same spirit — a conceptual model of one architecture, with the category membership question settled elsewhere.

Open Questions

  • Whether search reach can be measured or bounded for a socio-technical loop in the way oracle strength can be graded.
  • Whether a fallible evaluator can govern changes to its own acceptance criteria without either an external criterion or the axiomatization that buys formal closure.

Relevant Notes: