Reflective theory refinement has separate structural, epistemic, and implementation lineages

Type: kb/types/note.md · Tags: learning-theory, deploy-time-learning, self-improving-systems, foundations

No single predecessor is closest to reflective theory refinement, theory refinement of a theory about the learning system itself and earlier called theory-mediated system learning, on every relevant dimension. A similarity claim must say what it compares:

  • the object and target of representation — whether the retained model is about the modifying system's own structure, purposes, requirements, or goals;
  • the learning mechanism — whether an explicit, fallible theory guides inference and is itself revised by empirical contradiction; or
  • the implementation pattern — whether a contemporary LLM system learns by editing persistent code and text around a fixed model.

These dimensions identify different neighboring lineages. Collapsing them into one ranking makes a recent system look closest because its software stack resembles the current implementation, even when older work matches the scientific question more directly.

Runtime self-modeling and self-adaptation supply the structural lineage

Computational reflection introduced the central structural relation: a computational process operates through a causally connected representation of selected aspects of itself. Models@run.time developed runtime models of a system's structure, behavior, or goals as machinery for inspection and adaptation. Requirements reflection proposed making requirements first-class runtime entities so that a system could reason about, explain, and modify them while it operates. Architecture-based systems such as Rainbow used explicit architectural models to monitor and adapt a running system.

This family is closest to what the theory is about and what it changes. The represented object is the running system's own organization, requirements, or goals, and the representation lies on a causal path into changes to that system. That is closer to a retained program theory governing modification of the behavior-determining system than a theory whose primary object is an external environment.

The remaining gap is learning architecture. Classical runtime-model and self-adaptive systems usually begin with a supplied model language, goal or requirements vocabulary, monitoring boundary, adaptation operators, and evaluation policy. They need not treat the runtime model as a fallible explanatory theory whose content and scope are revised by delayed consequences, and they generally leave these adaptation-machinery choices outside the revision surface. Such choices belong to a software house's production machinery when they participate in the complete producer, whether or not the house uses a Greenfield-style factory. Their fixed placement limits claimed production reach when they carry required task- or product-scope-specific production knowledge. It is not an inherent defect when a component is warranted general machinery over the declared reach.

Theory refinement supplies the epistemic lineage

Explanation-based generalization is an established case of learning guided by an explicit theory: background theory determines what can be inferred from an experience rather than merely accompanying an empirical learner, though the theory itself is not revised. Theory refinement combining analytical and empirical methods goes closer to the present mechanism by beginning with an imperfect explicit theory, using empirical failures to identify candidate repair locations, and revising the theory rather than learning only from scratch.

This family is closest to how a fallible explicit theory learns. The theory shapes interpretation and search; evidence can count against the theory rather than only against one candidate answer; and repair can be localized to the represented knowledge implicated by the failure, with evaluation needed to check whether a proposed repair improves it.

Its usual target is nevertheless an external classification or problem domain, not the purposes and architecture of the learning system itself. Classical theory refinement therefore supplies the epistemic half of the proposed path without supplying its reflective self-target.

The mechanism is the same in both, slot for slot: a theory language, an interpreter that applies the theory, an evidence source, a failure localizer, repair operators, and an acceptance test. The proposed regime changes the fillers. The language is open-ended natural language and mixed forms rather than a fixed rule language, so what the theory implies is interpreted rather than proved, and only a codified part regains a computed consequence relation. One LLM fills the interpreter, localizer, and repair slots that a prover, abduction, and an induction algorithm filled. The evidence is production consequences rather than a labeled batch, which is why credit assignment and admission have less constrained inputs than in the classical systems. Those systems already needed diagnostic heuristics and candidate evaluation; explicit proofs did not guarantee a unique diagnosis or successful repair. And among revisions that fit the evidence, acceptance prefers explanatory reach, where the classical test was consistency with the training set. FORTE seeks a minimally revised theory correct on supplied cases, but its search can stop at a local maximum. Two differences are not fillers. A project theory is partly normative, so a failure may be resolved by changing the product to fit the theory, where the classical loop only ever changed the theory to fit the data. And the slots themselves, language, operators, and evaluator, are inside the revisable state, where the classical systems fixed them outside learning. What the field already had, and this program should not claim as new, is recurrent use of the refined theory, incremental refinement, localized repair, a bounded sample-efficiency result, and a human-engineered seed.

Workspace Optimization supplies a contemporary implementation analogy

Workspace Optimization is unusually close to the present implementation pattern. It keeps the foundation model fixed while making surrounding code and text editable, routes prediction failures toward responsible artifacts, and replays prior transitions after edits. Its persistent workspace is inspectable, executable, and consumed by later model calls.

That makes it a strong precedent for non-weight learning surfaces, local credit assignment, and counterexample-driven artifact repair. It does not make it the overall closest antecedent to the research program. Its explicit theory primarily models an external game environment within one run. Its role decomposition, validation, and adoption policy remain supplied, and the reported result does not establish cross-session recurrence of a revised theory of the behavior-determining system itself.

Workspace Optimization should therefore be described as a close contemporary LLM-agent implementation analogue, not ranked above the runtime self-modeling, self-adaptation, and theory-refinement traditions.

The proposed program joins the lineages

Reflective theory refinement proposes one causally co-indexed longitudinal path:

addressable theory of a software system's purposes and organization
  -> theory-guided search, diagnosis, and modification
  -> change to the behavior-determining system
  -> independent or delayed consequence
  -> read-back against the same theory
  -> explicit theory revision
  -> changed later modification

Runtime reflection and self-adaptation supply the self-model and causal target. Theory refinement supplies the fallible-theory and empirical-repair mechanism. LLMs supply a current semantic interpreter and search process over theories that have not been fully formalized. Persistent natural-language and symbolic artifacts provide addressable working state, while code and runtime carry exact transitions, checks, and continuity.

The further ambition is that required task- or product-scope-specific decompositions, evaluators, and improvement methods can be computationally acquired or challenged rather than supplied anew by people as the demand class widens. The ambition does not require every general algorithm, runtime, interface, or trusted kernel to modify itself. No source above establishes the complete evidence-to-production-knowledge transition, and the current program has not yet demonstrated it either. The combination is the research target, not a settled novelty result.

Consequences for comparison and experiment

Different neighbors imply different baselines:

  • a runtime self-model or architecture-based adaptation baseline tests what claim-addressable natural-language program theory adds beyond an explicit but designer-supplied system model;
  • a classical theory-refinement baseline, with its language, operators, and evaluator fixed, tests what the regime's fillers add: an open-ended theory language, reach-preferred acceptance, and a revisable update space;
  • a Workspace Optimization baseline tests the value of a program self-theory against editable external state, local failure attribution, and replay; and
  • direct search, parametric adaptation, meta-learning, and stronger models test whether explicit theory is needed at all for the observed recovery or transfer.

A result showing only persistent artifact editing, fast adaptation, modular reuse, or prediction-error correction would not identify the full mechanism. The discriminating result must connect theory use in the decision, change to the modifying system, empirical read-back, selective theory revision, and later changed modification on one traceable path.

Scope

  • This note positions the program; it is not a complete literature review or a priority claim over every historical source.
  • "Closest" is meaningful only after the comparison axis is stated. Structural precursor, epistemic mechanism, implementation analogue, and experimental competitor are different relations.
  • A runtime requirements or architectural model is not automatically a program theory in the present sense. The comparison concerns its self-target and causal role, not an identity of concepts.
  • A theory-refinement system is not automatically reflective. Explicit theory revision can concern a world outside the learner.
  • Supplied adaptation machinery bounds the demonstrated reach when it embeds target-specific production knowledge. Human provenance or fixedness alone does not show that scoped learning or computational closure is incomplete.
  • The proposed synthesis may turn out not to outperform simpler combinations of runtime models, direct search, artifact repair, and parametric adaptation.

Open Questions

  • What is the strongest implementation of requirements reflection or Models@run.time that can serve as a programming-agent baseline?
  • Can a theory-refinement treatment be made information- and compute-comparable to the proposed retained-theory treatment?
  • Does natural-language theory provide value beyond a structured runtime model when both expose the same purposes and dependencies?
  • Which part of Workspace Optimization's advantage comes from editable state, local ownership, replay, or theory-level organization?
  • Which supplied adaptation choices carry task-specific competence that the learning process must acquire, and which are warranted general machinery over the declared reach?

Relevant Notes: