Disconnected witnesses do not establish a full causal path through theory

Type: types/note.md · Tags: self-improving-systems, evaluation, theory-builder, learning-theory

A claim that a theory guided a decision, its consequence informed criticism, and the result guided later work concerns a connected causal path. Evidence that a theory existed, a decision occurred, an outcome followed, a theory changed, and later work used retained state does not by itself connect those events. The witnesses must identify the joins. Even a connected path through a theory builder establishes learning only when the process of conjecture and criticism improves the system's capacity for future action.

For an empirical recurrent claim, the path has this shape:

theory T0
  -> decision or search using what T0 says
  -> realized change
  -> independent or delayed consequence
  -> formulated criticism read against T0
  -> result affecting theory, reliance, or further test choices
  -> later operation using that result

The theory may be revised, rejected, or retained after a serious attempt to refute it. Survival can change reliance without changing theory text or its tentative status. The result can persist in a theory and testing record or in criticism used to reconstruct a theory. The path tracks the effect of criticism rather than requiring a new document version at each turn.

This shape composes three requirements. A complete theory path does not establish improved capacity separates mediation, empirical contact, response to criticism, and recurrent mediation. Citing retained theory at the decision point is a mediation trace identifies the theory a decision claims to have used, while leaving load-bearing use and criticism open. History has one chance to become checkable explains why facts needed to identify non-re-derivable joins must be carried forward while they remain available.

Each join supports a different inference

The first join identifies mediation: which theory content entered the decision whose behavior is attributed to theory. A retrieval log or later citation to a relevant theory need not identify what governed the decision. Withholding, replacement, or perturbation can supply stronger causal evidence, within the contrast actually run.

The middle joins identify empirical contact and response to criticism: which change produced the consequence, how that consequence challenged what the theory said, and what the criticism changed. A failure followed by an unrelated edit does not show that the failure revised the theory. Acceptance of a change does not verify the theory that motivated it. A recorded pass that affects neither later reliance nor later testing does not complete the path.

The final join identifies recurrence: whether the result of criticism affected later operation. Git ancestry can establish that work descends from an earlier commit. It cannot by itself establish which theory or criticism the later decision used, or whether its content was load-bearing. A reconstructed theory needs the same causal account of how retained criticism affected it.

The evidence levels compose only when their identities compose. Separate valid demonstrations of mediation, an outcome, a response to criticism, and later work can remain partial results. A reflective claim additionally needs the theory to be a causally connected self-representation of the same system aspects along the claimed path; self-description elsewhere in the project cannot supply that connection.

Recording follows from the evidential claim

For a record-based full-path claim, a reviewer must be able to associate the episode, supplied theory content, decision or search branch, realized change, consequence, criticism, its result, and later use. Where retained theory is versioned, versions can help identify the content. Where reconstruction is claimed, the evidence must instead identify what criticism was retained and how it informed the reconstructed theory. Actor or operator interventions must be retained where they determine a join.

Some deterministic or otherwise re-derivable joins can be reconstructed by replay. For nondeterministic LLM production, omitted prompts, supplied theory parts, rejected alternatives, or operator interventions generally cannot be recovered from the final artifact. Recording follows from relying on that history as evidence; it is not a membership condition of learning. Private linguistic formulation may occur without a visible record, leaving an observer unable to classify the process. Opacity alone establishes neither presence nor absence.

The exact event schema is an operational choice. Recording more fields cannot repair an experiment that never ran the contrast its causal claim requires.

Scope

  • The final observed-use link is required for a claim of observed recurrence, not for every claim of learning. Capacity can improve before an occasion for exercising it arises; a claim about later capacity requires persistence of the effect to that time, not an arbitrary minimum duration.
  • A learning claim must identify what capacity improved, by what criterion, and why the improvement is attributable to the process of conjecture and criticism. A fully connected path can still worsen decisions.
  • The diagram covers an empirical path. Formulated criticism by argument can also support learning without an intervention-outcome sequence.
  • Identity across witnesses does not establish that each link is causal. Interventions and appropriate controls are needed at the strength claimed.
  • The path can cross model, symbolic, environmental, and human components. Its identity implies neither one substrate nor computational autonomy.

Relevant Notes: