Gate learning from accepted edits
Type: kb/types/note.md · Status: seedling · Tags: kb-maintenance
The review system could learn new gates from human-accepted edits instead of accumulating lessons as ever-larger prompts. The central constraint rests on an accepted edit verifies the change, not the rule: accepted edits are the strongest available seed, but candidates mined from them are instance-verified, not rules — so the design must carry a promotion-and-rollback lifecycle, not just accumulation. This proposal holds the learning loop, the gate-lifecycle additions, and the loading policy.
Current state (as of 2026-06-12)
Shipped — proposed alongside this loop and since adopted:
- Atomic gates as single files with a type contract:
kb/instructions/review-gates/{lens}/{name}.md, typereview-gate. - Per-gate provenance, acceptance, and staleness:
gate_sha, per-gate acceptance events and current-acceptance views in the review SQLite ledger. - Selector machinery:
resolve_gates,review_target_selector,warn_selector. The gate-refactor workshop executed the move from monolithic review bundles to gate-native storage.
Not shipped:
- Any learning path from accepted edits to new gates.
- Per-gate precision/recall history; gate lifecycle states beyond existence.
- Retrieval- or budget-based gate loading beyond static gate resolution.
Raw material exists: review-revise experiments produced human-accepted before/after pairs with named changes (change catalogues in workshop runs) — exactly the capture format a mined candidate generalizes from.
The learning loop
- Capture the pre-edit note, post-edit note, and accepted diff.
- Segment the diff into edit intentions, not line changes.
- Compare intentions against the gates that were loaded for the originating review.
- For uncovered intentions, propose candidate gates.
- For covered intentions, update evidence that the existing gate still earns its place.
- A human accepts, merges, rewrites, or rejects each candidate.
The oracle stays where it is strongest: "did a human accept this change," not "was the critique clever." Step 6 is the promotion gate the oracle asymmetry requires.
Gate lifecycle additions
Each gate gains: status (candidate / active / quarantined / retired), provenance (which accepted edits produced or confirmed it), and a precision/recall history per run. A noisy gate is quarantined without rewriting the rest of the review method; a consistently useful one can mature toward a deterministic check — the methodology enforcement gradient. The registry stays inspectable substrate rather than prompt folklore.
Loading policy
A fixed token budget beats a fixed gate count — some gates are one line, others need a counterexample. The selector fills the budget from: always-on base gates (cheap, high-precision), lens-specific gates, retrieved gates (matched to the note by tags, vocabulary, type, prior similar failures), with diversity pressure against spending the budget on near-duplicates. Rests on instruction specificity should match loading frequency: always-loaded review instructions stay slim, specificity moves into on-demand gate loading.
Risks
- Attribution ambiguity in coupled edits (a structural rewrite also fixes clarity — which gate earned the acceptance?).
- Candidate gates overfitting one author's style or one note family.
- Revisions that satisfy a gate while worsening the note elsewhere.
- Observed in the seed experiments: review batteries missed structural changes and once induced a factual error — rollback is load-bearing, not optional.
Open questions
- The minimal gate schema that still supports pruning and promotion.
- Where candidates live (a workshop area?) versus active gates (
kb/instructions/review-gates/). - The promotion metric: precision alone, stable recurrence, or measured contribution to accepted revisions — composite weak signals are a candidate (quality signals for KB evaluation).
- Whether revision should see raw gate text or only the findings emitted from gates.
- Negative gates: mining patterns that looked useful once but later caused mistakes.
Relevant Notes:
- an accepted edit verifies the change, not the rule — rationale: the oracle asymmetry that dictates candidate status, promotion, and rollback
- spec-mining-as-codification — rationale: mining reusable checks from observed accepted behavior is the general move this design instantiates
- instruction specificity should match loading frequency — rationale: budget-bounded selective loading instead of always-growing review prompts
- methodology-enforcement-is-constraining — rationale: the maturation gradient gates climb from candidate toward deterministic check
- review architecture — part-of: the shipped review subsystem this loop would extend