Bottom-up structure inference needs capture at the decision surface, not the state

Type: kb/types/note.md · Status: seedling · Tags: agent-memory, context-engineering, learning-theory

Trace-derived schemas, ontologies, and memory all promise the same thing: instead of prescribing a world model upfront, infer the minimal set of entities and relations from what actually happened. But that inference is not free-floating over "traces" in general. It is gated by where in the workflow capture happens. Bottom-up structure inference is feasible only when what is captured is decision-shaped — the inputs referenced, the constraints in play, the exception path, the approval, the action taken, the outcome — rather than state-shaped: the row that changed, the final value, the resulting record. The reason is a locality property of intent: the "why" behind a change is maximally available at the moment of decision and decays toward unrecoverable in the state the decision produces.

The "why" is cheap at the decision surface and expensive-to-impossible from state

At the instant a decision is made, the deciding process is holding the material that explains it. Recording those "decision receipts" at that surface is nearly free, because the information is already in hand — to the extent the decider can articulate its why: an opaque decider (a gut call, a neural policy) holds less in articulable form, and its receipts are correspondingly thinner. The resulting state, by contrast, keeps only the endpoint. A changed field tells you that something is now true; it does not tell you why — which piece of that material actually decided it. Reconstructing that after the fact means re-deriving intent from residue — sometimes expensive, often impossible. This is the same gap raw accumulation does not create usable memory names as the ingress problem: you can have state, but not the legible "why," and retrieval cannot repair a "why" that capture never preserved.

Capture position is therefore a precondition on the inference mechanism

The mechanism that turns observed behavior into structure — spec mining, codification's operational mechanism — reads regularities off a stream of observations: rules and checks in spec mining's home cases, entities and relations here. That mechanism can only recover structure that the stream actually carries. If the stream is state-shaped, the entities and relations it can support are the ones visible in endpoints; the intent-level structure (why decisions branch, which constraints bind, what an exception means) is simply absent from the input and cannot be mined. So the feasibility of bottom-up inference is decided upstream of the inference algorithm, at the capture point: instrumenting decision surfaces rather than data stores changes which world models are learnable, not just how efficiently they are learned, because structure whose evidence never entered the stream cannot be mined out of it at any cost. Deferring structure is safe, per the wikiwiki principle; deferring capture of the why past the decision surface is not, because the why does not wait.

This claim is about rationale-bearing structure specifically — the entities and relations that name why a case went one way (which policy bound it, which exception fired, who approved it) — not just any regularity a learner can extract from a stream. Population methods that mine decision rules from event logs (process mining, decision mining, dynamic-invariant detection) recover predictive rules over case attributes already present as fields in the log; that is fitting statistics over an existing vocabulary, not inferring the vocabulary.

Boundary: right capture point is necessary, not sufficient

Positioning capture at the decision surface makes bottom-up structure possible; it does not make the captured receipts true. A decision receipt is a record of what the deciding process attests happened — and an attested why can be confabulated — so trace-derived memory earns authority per operation, not at capture: the inferred entities and relations remain guesses until verified, and stall out as structure-shaped noise if the store never climbs past capture. Together, this note and that one name two independent preconditions on trace-derived memory — capture must be positioned to hold the why (this note), and the derived structure must be verified to hold authority (that note). Missing either sinks the approach.

Open Questions

  • How decision-shaped must capture be? There is presumably a spectrum between raw state and a fully structured receipt; the claim only requires enough of the why to be present for the target structure to be inferable, but where that threshold sits is likely task- and ontology-specific.
  • Can partial "why" be reconstructed from rich enough event logs (not just final state)? If so, the dichotomy is really a continuum along how close the capture point sits to the decision, and "state vs decision" names its two ends.
  • Population methods that go beyond field statistics already exist: inverse reinforcement learning and inverse constraint learning recover latent reward functions and binding constraints that appear nowhere as fields in a state-action trajectory. Whether that counts as inferring a rationale vocabulary — and what operational test (counterfactual validity, out-of-distribution transfer) would decide it — is the sharpest version of the continuum question above; if it does count, this note's "only if" needs narrowing.

Relevant Notes: