A retained-theory intervention isolates one explicit theory surface, not the system's whole program theory

Type: kb/types/note.md · Tags: foundations, self-improving-systems, evaluation

A system may carry program-relevant understanding across several places at once: explicit explanations, architectural decisions, code, tests, operational state, model competence, and human participants inside the declared boundary. A retained-theory intervention varies a declared explicit theory state while holding specified background components fixed. Its result therefore identifies the causal contribution of that theory surface under the intervention; it does not remove every theory-bearing influence from the composite or test whether the composite possesses a complete program theory.

This distinction matters because explicit retained theory is unusually convenient to manipulate. Its claims can be retrieved, cited, withheld, replaced, perturbed, or selectively revised, since only explicit retention is currently durable, writable, and addressable at once. But addressability does not make that surface exhaustive. Code and tests may encode related commitments symbolically, model weights may supply learned competence that reconstructs similar judgments, and an operator may carry relevant understanding that was never externalized.

What the intervention identifies

Suppose two runs hold the model, symbolic state, tools, task, and budget fixed while varying only the retained theory supplied to the model-mediated decision process. A difference in proposal, branch allocation, diagnosis, evaluation, recovery, or later revision is evidence that the varied theory surface mediated that difference. Withholding, replacement, or targeted perturbation is stronger evidence than merely observing that the theory was retrieved or cited, because citing retained theory at the decision point is only a mediation trace.

The identified contrast remains local to the treatment. If the withheld theory can be reconstructed from code, tests, task wording, model competence, or another retained artifact, a null result does not show that program theory is useless. It shows that varying this declared surface under these background conditions did not change the measured operation. Conversely, a large effect does not show that the surface contains the whole program theory; it shows that the information or organization varied there was causally important in this comparison, since an experiment identifies only the contrast it actually runs.

Retained theory state is an experimental object

For an intervention, retained theory state should name the versioned, claim-bearing theory input designated as part of the treatment. That is narrower than all retained project state and narrower than everything the composite may in a dispositional sense know about the program. The distinction makes the treatment reproducible enough to state: a protocol can name which theory version was supplied, which components were fixed, and which outcomes were compared.

This experimental object also differs from the wider deployed-system learning unit. The deployed system can learn by changing prompts, code, tests, schemas, tools, runtime policy, or other retained state. A retained-theory intervention asks a narrower causal question inside that larger learning surface: what difference does this addressable theory state make to the improvement path?

Scope

  • The claim concerns causal interpretation of interventions on explicit retained theory. It does not define all forms in which a system may possess or reconstruct program-relevant understanding.
  • Holding symbolic state fixed is a protocol choice for isolating the theory surface, not a claim that symbolic artifacts cannot themselves encode theory-bearing commitments.
  • Information-matched controls may be needed to distinguish the effect of theory organization from the effect of supplying additional task-relevant information.
  • A null result against one retained-theory surface does not establish that generic search and theory-guided search are equivalent under other representations or task distributions.

Relevant Notes: