review-system

Type: types/tag-readme.md

Semantic review of KB artifacts by an LLM: what a review can catch that validation cannot, when a review verdict goes stale, what reviews cost, and proposed changes to the review pipeline. The shipped system is described in the review system reference, which defines its concepts: a review judges a note against a criterion file; a gate is a closed-ended criterion whose outcome is pass, warn, or fail; a result file keeps the judgment with the hashes of both texts, and stays fresh while they are unchanged. Deterministic structural checks (schemas, links, required sections) are not review; they belong to the validation side of document-system. A child of kb-maintenance.

What review checks and when verdicts go stale

• Grounding versus representation — distinct semantic checks apply to sources and linked claims
• Narrowing can empty a reviewed claim — gate success needs a separate check of surviving contribution
• Reasoning production versus evaluation — review must inspect the stated argument rather than reconstruct an answer
• Reviewers share the field's prior — read semantic findings against correlated priors and the claim's stance
• Verifiable review subroles — narrow evidence targets precede reviewer-level authority
• A five-link cap missed four grounding findings in twelve reviews — paired reviews test how the linked-reading budget changes which grounding errors a gate detects
• Semantic review catches content errors that structural validation cannot — why review exists beside validation: enumeration completeness, grounding drift, boundary cases, internal contradictions
• A note is an atomic step relative to the check that reads it — sizes a note to one inference a reviewer can check in a single pass
• Criteria edits invalidate verdicts; process edits invalidate artifacts — which edits make a verdict stale: the note and the criterion count, the production process does not
• Full improvement pass closure — how the shipped full-improvement workflow reassays final note bytes, routes residual findings, and stops without claiming convergence
• Single-artifact review bundles still cut Claude costs substantially — cost evidence: the single-artifact bundle refactor held its savings after cache-aware price weighting

Proposals: freshness and invalidation

• Factored dependency pairs for review freshness — proposal to keep review dependencies as two-input pairs where that shape fits; decides how review targets with more than two inputs are handled
• Collection-as-artifact freshness — proposal to register collection-maintenance targets with collection-text inputs, for casebook-wide staleness without per-file dependency edges

Proposals: gates and review method

• Calibrating semantic gates against labelled fixtures — proposal to require known-case regression before a gate advances, and to keep live detection-rate claims for separate field calibration
• Gate learning from accepted edits — proposal to mine gate candidates from accepted note edits with a promotion lifecycle; atomic gates shipped, the learning has not
• Routine bilateral isolation for literature assessment — proposal deciding whether matched evidence should make bilateral isolation a routine control rather than a conditional diagnostic
• Automated note refinement as a search over a fixed source bundle — proposal to refine from a source bundle that emits a set of notes, so split, drift, and kill become search outcomes rather than non-convergence

Proposals: review machinery

• Structured-output codec for the review protocol — proposal to make review output a pluggable codec: sentinel markdown now, schema-validated structured output when harnesses support it

Related Tags

• kb-maintenance — the parent: review is one of the detection mechanisms that keep the KB healthy
• claims-and-grounding — sibling: grounding is the most common thing a gate checks, and its evidence notes come from review runs
• curation — sibling: index and tag upkeep, which review does not judge
• evaluation — how LLM judgments are calibrated and trusted, which gates depend on
• observability — staleness signals and review telemetry
• document-system — the deterministic validation that review complements