review-system

Type: types/tag-readme.md

Semantic review of KB artifacts by an LLM: what a review can catch that validation cannot, when a review verdict goes stale, what reviews cost, and proposed changes to the review pipeline. The shipped system is described in the review system reference, which defines its concepts: an assay applies a criterion to a note; a gate is a closed-ended, verdict-kind criterion whose outcome is pass, warn, or fail; a freshness baseline records when a verdict still applies. Deterministic structural checks (schemas, links, required sections) are not review; they belong to the validation side of document-system. A child of kb-maintenance.

What review checks and when verdicts go stale

• Semantic review catches content errors that structural validation cannot — why review exists beside validation: enumeration completeness, grounding drift, boundary cases, internal contradictions
• A note is an atomic step relative to the check that reads it — sizes a note to one inference a reviewer can check in a single pass
• Criteria edits invalidate verdicts; process edits invalidate artifacts — which edits make a verdict stale: the note and the criterion count, the production process does not
• Full improvement pass closure — how the shipped full-improvement workflow reassays final note bytes, routes residual findings, and stops without claiming convergence
• Single-artifact review bundles still cut Claude costs substantially — cost evidence: the single-artifact bundle refactor held its savings after cache-aware price weighting

Proposals: freshness and invalidation

• Factored dependency pairs for review freshness — proposal to keep review dependencies as two-input pairs where that shape fits; decides how review targets with more than two inputs are handled
• Collection-as-artifact freshness — proposal to register collection-maintenance targets with collection-text inputs, for casebook-wide staleness without per-file dependency edges
• Generalized validation invalidation and imperative extension — proposal deciding when to generalize validator invalidation or imperative type extension: only after local cases prove reusable machinery

Proposals: gates and review method

• Calibrating semantic gates against labelled fixtures — proposal to require known-case regression before a gate advances, and to keep live detection-rate claims for separate field calibration
• Gate learning from accepted edits — proposal to mine gate candidates from accepted note edits with a promotion lifecycle; atomic gates shipped, the learning has not
• Routine bilateral isolation for literature assessment — proposal deciding whether matched evidence should make bilateral isolation a routine control rather than a conditional diagnostic
• Automated note refinement as a search over a fixed source bundle — proposal to refine from a source bundle that emits a set of notes, so split, drift, and kill become search outcomes rather than non-convergence

Proposals: review machinery

• Model partition registry — proposal for a registry of model partitions for validation, aliases, and runner defaults, without making it the review identity
• Review configuration object — proposal for a ReviewConfig holding scan roots, gate and artifact locations, and database path, with project overrides
• Structured-output codec for the review protocol — proposal to make review output a pluggable codec: sentinel markdown now, schema-validated structured output when harnesses support it

Related Tags

• kb-maintenance — the parent: review is one of the detection mechanisms that keep the KB healthy
• claims-and-grounding — sibling: grounding is the most common thing a gate checks, and its evidence notes come from review runs
• curation — sibling: index and tag upkeep, which review does not judge
• evaluation — how LLM judgments are calibrated and trusted, which gates depend on
• observability — staleness signals and review telemetry
• document-system — the deterministic validation that review complements