Theory builder

Type: types/definition.md · Tags: self-improving-systems, learning-theory, theory-builder

A theory builder is a system that applies Popper's method of conjecture and refutation to theories it keeps as objective knowledge. It works through Popper's schema P1 → TT → EE → P2: faced with a problem, it proposes a tentative theory, attempts to eliminate its errors, "especially by way of critical discussion", and takes up the new problems that result (Popper 1966). The KB needs the term to name the kind of system Commonplace builds and studies.

A system is a theory builder when it meets four conditions, each grounded in Popper's account.

  1. Localized content. Its theories are stated in natural or formal language, so identifiable units carry their content: each unit says something that can be pointed at. This is the localized side of representational form. Popper's objective knowledge is knowledge "contained in a book; or stored in a library" (Popper 1966), and he holds that criticism needs this form: without a descriptive language "there can be no object for our critical discussion" (Popper 1968). How finely the units divide a theory is graded (see Addressability).
  2. Consumption. Its theories guide what it does through what they say. In Popper's words, "all our actions in the first world are influenced by our second-world grasp of the third world" (Popper 1968). A difference in a theory's content that matters to a decision changes the decision (operative change).
  3. Criticism. It has a working process of attempted refutation aimed at what identified units say: critical argument, comparison with rivals, and tests of stated consequences. A criticism is itself stated, so it can be criticized in turn; in particular, it can blame the test, the data, or an auxiliary assumption instead of the theory. Theories that fail are revised, or rejected whole and replaced by a new conjecture: for Popper, our consumption of theories "also means criticising them, changing them, and often even demolishing them, in order to replace them by better ones" (Popper 1966). Revision need not be small: it may change a core assumption, the problem, or the machinery.
  4. Iteration. The result of criticism is kept and shapes the next round. In Popper's schema, "the result of criticism, or of error-elimination, applied to a tentative theory, is as a rule the emergence of a new problem" (Popper 1966), and that problem P2 is the starting point of the next round. After a refutation, the next round works with a revised or replacement theory; a theory rebuilt from retained criticism is a new conjecture informed by it. A revised theory is a new conjecture, so testing it again on cases that refuted its predecessor is a real test. After a survival, the text may stay the same while the theory's testing record changes. The record is part of what the next round works with. Popper notes "something like a law of diminishing returns from repeated tests (as opposed to tests which, in the light of our background knowledge, are of a new kind)" (Conjectures and Refutations, Chapter 10), so the problem becomes finding a test of a new kind, and the record also changes how far the theory is relied on. A change to the record counts only when later work uses it. How long and how widely results persist is graded (see Persistence).

Criticism against gradient descent. Gradient descent also eliminates error, and it assigns blame more finely than any text: every parameter gets its share. But no parameter says anything by itself, so the blame cannot be stated as an error in what the theory says, and it cannot be argued with; the loss and the data are fixed from outside the process. This is Popper's distinction between the critical method and trial and error, which is applied "in a more dogmatic fashion, by the amoeba also"; the difference "lies not so much in the trials as in a critical and constructive attitude towards errors" (Conjectures and Refutations, Chapter 1). Conditions 1 and 3 together draw this line: localization supplies units that say something, and criticism aims at what they say.

Error elimination is attempted, not guaranteed. The parentheses in Popper's "(attempted) error-elimination" are his. A builder whose criticism finds nothing, or whose revisions do not improve later work, is still a theory builder. Whether a theory builder learns, in the sense of improving its capacity for future action (Simon's criterion), is an empirical question about it. The definition does not settle it.

Addressability

Condition 1 sets the minimum: some unit carries content, even if that unit is the whole theory. Above the minimum, addressability comes in grades: the finer the units, the more precisely criticism can name what it blames. At the high end, a theory's assumptions, scope conditions, and parts are stated separately and can be revised individually. Popper grants Duhem that a test often bears on a whole system, but replies: "It is possible in quite a few cases to find which hypothesis is responsible for the refutation" (Conjectures and Refutations, Chapter 10). Commonplace builds for the high end. That this pays is a conjecture, tested against builders whose theories are coarser; the lowest baseline states each theory as one undivided unit.

Addressability is relative to the unit criticism names, and this applies to the machinery as well as to the theories. A model is not localized inside, but as a component of the builder it is an addressable part: a method text or configuration states which model does which work, criticism can blame that choice, and the builder can replace the model whole. What the builder cannot do is criticize what the model's weights say, because no unit in them says anything.

Persistence

Condition 4 sets the minimum: the result of criticism reaches the next round. Above the minimum, persistence comes in grades: within one reasoning episode, across the rounds of one run, across runs on the same task, and across problems and sessions, where later work on other questions starts from retained theories and their testing record. Commonplace builds for the high end: a library of retained theories that later work takes up. That this pays is a conjecture, tested against builders whose results persist less; the lowest baseline keeps only records of inputs and outcomes, which carry nothing criticism produced.

When a run freezes a theory and another system deploys it, the deployment does not extend the builder. The builder stopped at the freeze; what it was while it ran is unchanged.

Boundary

The builder is the whole system that performs the operations above: people, models, tools, and retained texts. The operations are noticing a problem in its own theories or work, proposing a theory, deriving what it implies, criticizing it, choosing what to blame, producing a revision, selecting the theory to keep, and changing the machinery that does these things. Noticing starts the cycle: a recurring failure, a theory that has gone stale, or a connection nobody made becomes a problem the builder takes up. Users who supply problems as tasks and judge the products are outside the builder unless they perform one of these operations. The boundary follows the operation, not the person, so a claim about a builder declares the boundary it assumes. Selecting what to keep is one of the operations: a model may propose and outside evaluators may assess, but what carries into later work is decided by the builder's process.

The builder is identified by its continuing process, not by any component. Every theory, procedure, and model in it may be replaced over time through its own criticism and revision. A change installed from outside that process is an intervention and is recorded as one.

Qualifiers

  • Reflective. The builder's method is part of its objective knowledge. The problems it works on, its standards of criticism, and its procedures meet the four conditions, as its other theories do. Popper places these objects in the third world: only there "the problems and standards of rational criticism can develop" (Popper 1968). A procedure is a tentative solution to a problem about how to build theories. Criticism aims at its stated conjecture about why it works, so a procedure with no stated purpose can be tried but not criticized. The causal connection that reflective system requires runs through conditions 2 and 3: the builder's operations consume the method texts, and criticism tests those texts against records of the builder's own operation. Popper describes the human version as "the give and take between ourselves and our work", with "feed-back that can be amplified by self-criticism" (Popper 1968). Unlike computational reflection, this connection is not kept up automatically. When the machinery changes outside the texts, for example when a model is replaced, text and operation can diverge until criticism finds the gap. Reflection reaches a model only as an addressable part (see Addressability): the builder can hold theories about the model and criticize the choice of it, but it changes the model only through what it gives the model or by replacing it.
  • Autonomous. Computation performs every operation inside the boundary. Users still supply problems and judge products. Autonomy does not establish that the operations are reliable.

The qualifiers are independent. Commonplace today is a reflective, human-staffed theory builder; the research program's bet is an autonomous one that learns. Reflection is structural: it makes the method open to criticism and implies neither learning nor compounding, where method changes make later improvement better. A system can also improve its own improvement machinery without reflection, by selecting method variants on score.

Boundary cases

  • A research community is a theory builder, with the community as the declared system. It is Popper's own case. No single member holds the whole theory or supplies all the criticism.
  • Commonplace's note-review loop, with the operator performing internal operations, is a human-staffed theory builder.
  • A refinement run such as FORTE is a theory builder at a low grade of persistence. Its Horn-clause theory is stated and highly addressable, misclassified examples refute it, proof traces assign blame to clauses, and each repair is tested again. What it lacks is persistence beyond the run.
  • A theory built while reasoning and then discarded is inside at the lowest grade when the reasoning states the theory, criticizes it, and revises it in response. Discarding it afterwards ends that builder.
  • Criticism applied through weights. Critique-trained reinforcement learning and "textual gradient" methods state a criticism, then use it to update weights. The revised theory is the weights, where no unit says anything, so the arrangement is outside. It becomes a builder only when the criticism aims at a stated theory that the system retains and consumes.
  • Content located in weights. Model editing and interpretability methods can find weights that carry a particular fact. To that extent those parts of a model move toward the localized side, and criticism aimed at them can meet condition 3. The definition tracks localization, not substrate, so whether a given model's weights meet condition 1 is an empirical question about that model and method.
  • The Gödel machine stays open. Its switching is governed by proof from premises that the construction does not criticize; whether a deployment criticizes them elsewhere is not settled by the construction. See Gödel machines are a proof-governed case of self-modification.

Exclusions

  • Dispositions and weight adaptation. Popper counts expectations and dispositions as tentative theories in a wider sense. Their content is not localized, so they fail condition 1 (for content located in particular weights, see Boundary cases), and adjusting them by gradient fails condition 3. A system whose only change is weight adaptation is not a theory builder. Its models can still be components of one.
  • Black-box optimization. Variants of prompts or programs are generated and kept by outcome score, with no stated reason bearing on what a variant says. The variants are localized, but selection does not aim at what they say, so this is trial and error and fails condition 3. Real systems fall between this case and a builder; the test is whether a stated reason bears on what a unit says.
  • A fixed theory. A stated theory guides decisions and the system never criticizes it. It fails condition 3.
  • Criticism that feeds nothing. A critic reports errors, but no next conjecture takes the report up: the output is ranked, filtered, or returned as it was. It fails condition 4.
  • A stored theory nothing consumes. It fails condition 2. See an action model matters only through its consumption path.

Misuse Cases

  • Calling a model, a prompt, a harness, or a review pipeline a theory builder when it is one component of the system that performs the operations.
  • Counting a system as a theory builder because it stores prose about its subject or itself. Storage satisfies condition 1 at most.
  • Reading membership as evidence of learning. A builder can criticize and revise without improving; improvement needs its own comparison.
  • Counting a user inside the builder because their problems or verdicts changed a theory, when they performed none of the internal operations.

Relevant Notes: