Structured-output codec for the review protocol

Type: reference/types/design-proposal.md · Tags: kb-maintenance, review-system

The review protocol's output side is one encoding: sentinel-delimited markdown blocks parsed by a hand-written state machine, with the outcome recovered from a strict final result footer. Harnesses are starting to offer schema-validated structured output for sub-agent calls — Claude Code's dynamic workflows agent(prompt, {schema}) forces a validated structured-output tool call and returns a parsed object, with validation retried at the tool-call layer. If that capability generalizes across harnesses, the remaining free-text failure classes disappear at the source. This proposal holds the design for making the output encoding a codec choice rather than a protocol property.

Current state (as of 2026-07-01)

  • (2026-11-10) ADR 127 replaced the sentinel-block protocol described in the rest of this proposal. commonplace-review run gates launches headless workers; each review is its own output file, so no sentinel blocks or job-output parser remain. src/commonplace/lib/review_workflows/outcomes.py requires exactly one final line, ## Result: PASS|WARN|FAIL for a verdict criterion or ## Result: REPORT for the critique, and rejects aliases and extra result-like lines. A review that fails the check is a rejected attempt whose problem goes into the next prompt, up to three attempts. A worker that cannot judge writes a problem file instead of an ERROR outcome.
  • The output contract lives in the prompt and check of src/commonplace/lib/review_workflows/gates.py. An accepted review is written as the pair's result file with the parsed outcome in its frontmatter and the reviewer's markdown as its body; the fix procedure reads findings from that body's ### Findings section.
  • The headless runtime also offers decide(), an ask whose answer is parsed and checked against a schema and closed choices; the full improvement pass uses it for its packet. Gate reviews do not use it.
  • Trigger not yet met: schema-validated sub-agent output ships in one harness's workflow scripts; the subprocess CLIs (claude -p, codex exec) and the live-agent file-artifact path have no equivalent surface the review system could consume today.

The design

A codec is the pair (render the output contract into the prompt, decode raw output into ParsedJobOutput). Two codecs:

  • markdown-sentinel (today): contract rendered as sentinel instructions plus a block template; decoder is the existing parser plus outcome protocol.
  • structured (new): contract expressed as a JSON schema — per pair: note path, gate id, summary, findings (severity + text), optional suggested revision, outcome as an enum. The decoder validates and maps to ParsedJobOutput; the outcome parser chain is bypassed entirely because the outcome arrives as a constrained field. The per-pair markdown result file can be rendered from the structured fields, so warning extraction and human review still use the derived result artifact boundary.

ParsedJobOutput is already the codec-independent boundary type. Missing pairs and structural errors both translate to failed live finalization under the all-or-nothing policy; a structured decoder should still distinguish them so callers can report the cause.

Free choices

  • Where codec selection lives. A flag on prepare/ingest commands chosen by the orchestrator (it knows whether its harness supports schemas), a per-runner-adapter property, or a project-level default. The orchestrator-chooses option matches the medium-pluggability direction carried from ADR 030 through ADR 035: the parent agent or harness owns dispatch and fan-out, Commonplace owns state and parsing.
  • Schema shape for findings. Mirror the current markdown sections one-to-one (summary/findings/revision) or take the opportunity to constrain severity to an enum and findings to a list — more validation power, but historical rationale text and the new format diverge in expressiveness.
  • Canonical retained form. Keep the markdown result file as the single retained review-body representation (structured output rendered to markdown on ingest), or add a new structured artifact alongside it. The first keeps one read path; the second preserves machine-readable findings for the gate-statistics ambitions in gate learning from accepted edits.
  • Whether the markdown codec ever retires. Free-text markdown is the lowest common denominator every harness supports; retiring it would couple the review system to schema-capable harnesses.

Adoption criteria

Adopt when a harness medium the project actually uses for reviews exposes schema-validated output at a surface the system can consume (a workflow orchestrating review batches, or a subprocess CLI flag). Adopting earlier buys nothing: the markdown codec must stay regardless, and the structured decoder cannot be integration-tested without the capability.

Risks

  • Two codecs mean two prompt contracts to keep semantically aligned; the codec interface must own both sides (render + decode) so a contract change cannot touch one encoding only.
  • Schema validation failures look terminal but may be retryable at the harness layer; the decoder should distinguish "harness delivered invalid object" from "model omitted a pair" for diagnostics even though both fail live finalization.
  • Findings-as-enum tightens what reviewers can express; the markdown codec's prose findings have carried nuance (multi-severity bullets, inline suggested rewrites) that a first schema draft may flatten.

Relevant Notes: