Retaining the episode keeps a distilled rule re-derivable

Type: kb/types/note.md · Tags: agent-memory, learning-theory, context-engineering

A lesson learned in operation admits two explicit retention forms. The episode is the trace of the situation the lesson came from — a session transcript, an execution trace, a worked case. The rule is the distilled statement of the lesson, separated from its occasion. Memory designs tend to treat these as rivals — episodic stores versus extracted facts — but they are layers of one system, and the choice that matters is not which to keep but whether the pair stays linked.

Linked, the pair instantiates a two-layer execution system at artifact scale: the rule is the derived fast path — action-shaped, cheap to load, strictly narrower than what the episode contains — and the episode is the generator-side evidence a consumer drops back to when the rule's coverage fails or its scope is contested. That fallback is what the title names. A challenged rule with its episode retained and lineage recorded can be re-derived: read the episode again, judge whether the generalization survives, revise its scope from evidence. This is semantic re-derivation under the managed-staleness regime, not deterministic recomputation — but it is real recourse. A rule whose episode is gone has none: it can be trusted or discarded, never re-examined against what taught it. Discarding the episode converts the rule from an evidence-backed derivation into a bare commitment, and an upstream change can then name no downstream worklist because the dependency record died with the source.

Distillation is earned by recurrence

The timing of distillation is a lifecycle question, not a storage preference. A rule distilled from a single episode is a conjecture that has skipped its test: the discovery lifecycle places one surprising case at observation and the posited generalization at conjecture, with acceptance gated on accumulated cases. Retaining the episode first and distilling on recurrence respects those phases — the episode store is where candidate generalizations wait for their second and third occurrence, and recurrence is the same promotion signal the two-layer architecture uses to grow a fast path. Distill-on-first-occurrence fixes a generalization exactly when the evidence for its scope is thinnest.

What distillation sheds, and where each layer wins

Distillation keeps the articulable part of a lesson and sheds the rest. The residue — calibration, situational feel, what a counterpart means by their words — is competence the episode still carries latently, because replaying an episode into context partially re-induces the conditioned state that held the lesson, while no statement of the rule can. The episode is the explicit trace of a tacit state, and the loss direction follows from only explicit retention being durable, writable, and addressable at once: the rule is the more addressable object, the episode preserves more of what resisted articulation. The practical familiar form of this asymmetry is that worked examples routinely outperform stated instructions for style- and calibration-shaped competence.

Governability runs the other way. Rules collide detectably — two contradictory statements can be noticed at write time — and are individually citable, revisable, and retirable. Episodes teaching opposite lessons coexist silently, and no lesson inside an episode can be revised; it can only be annotated or re-distilled. So governance lives at the rule layer, evidence and residue at the episode layer — a division of labor, not a contest. Between the raw trace and the bare rule sits a spectrum of intermediate forms — the cleaned worked trace, the rule with its attached example — and the residue share of a lesson's value is a guide to where on that spectrum it should be retained.

Costs the pair must manage

Scope

  • Which episodes to keep at all is the inclusion question, and it belongs to the declared output spec; this note owns the form question — given a lesson worth keeping, in which layer its value survives.
  • Nothing here claims raw transcripts are the right episode form; the claim is that some episode-grade record must survive distillation for the rule to remain re-derivable.

Open Questions

  • Eviction: episodes accumulate linearly with operation; what retires one — the promotion of its lesson, a staleness horizon, or contradiction by later episodes?
  • Recall: the rule layer is findable by statement; what routing lets an agent find the episode it needs when the rule's scope fails?
  • How much residue actually survives replay, and how it degrades across model versions, is measurable and unmeasured.

Relevant Notes: