Notes bearing on the analysis design
A sweep of every note in kb/notes/ and its definitions on 2026-10-10,
asking which claims bear on the engine, the compact plan, the prompts, the
correction and verification protocol and the records. Five scouts read
descriptions for all 392 files and bodies for about 70; this page keeps
what survived a second reading. Each row names the claim, the design
element it touches, and whether the design follows it, could use it, or
should watch it. Nothing here is decided.
Opportunities, in the order worth taking
| Note | Claim | Touchpoint | What it suggests |
|---|---|---|---|
| error messages that teach | the error channel is an instruction channel; a refusal that states the fix spares the agent the diagnosis at almost no cost | refusal findings and blockers | the validator's findings carry a repair text already; make the check's packet and the apply job's refusal always carry it, and make a blocker say what a corrected or declined answer needs |
| enforcement without structured recovery | detection without recovery is incomplete; recovery runs corrective, fallback, escalation | max attempts, exhausted jobs | the engine's only declared recovery is the corrective loop and a stop on exhaustion; name the escalation path in the plan and the driver: what the coordinator does with an exhausted job, and that nothing publishes |
| reasoning production is not reasoning evaluation | a model that can produce a good answer rubber-stamps a matching conclusion; review prompts must force inspection of the route | the three verifier instructions | require the verifier to inspect the cited route, record by record, and forbid agreement with the conclusion as grounds; a blocker-free verdict accepts every subject, so a lazy verifier accepts everything |
| reviewers share the field's prior | judges converge on consensus where a claim is original; the author's stance is a key for reading verdicts, recorded before review, not shown to the judge | the answers file's declined entries |
the declined path is where stance lives; keep verifiers from seeing the answers before judging, and read a declined blocker as a stance, not a defect |
| llm output deviation needs three-way diagnosis | underspecification, interpreter failure and sampling spread have different repairs; failure frequency alone does not pick one | repeated refusals, max attempts | a blocker could say which it is: spec violation, retry; spec gap, fix the type or mission. An identical failure on rerun is a spec signal and should escalate, not loop |
| silent disambiguation | a worker that silently resolves an ambiguity the spec left open hides that the contract failed to determine the path | missions, member types, verdict Limits | give the worker a channel to flag an undetermined point without failing, such as a Limits-like section in reports, rather than relying on gates to notice |
| oracle accumulation and progressive constraining | a retained check improves every later candidate; but freeze a pattern only after it recurs across runs | the consumer's declared checks | promote a recurring verifier blocker class into a code check or type rule, and only after it recurs; never into mission prose |
| full identity keys | key each unit of a batched protocol by full composite identity, not position | verdict subjects, problem lines, coverage | a blocker and a problem line should key by role plus record or relation, which the record-link form now gives for free |
| mixed epistemic status below the document level | status must be tracked per claim, not per document | records, the epistemic report, Limits | per-record status is a type-level field the validator could check; today status lives in prose and in document-level verdicts |
| semantic sub-goals that exceed one window | a judgment larger than one window is partitioned, with symbolic collection between parts | the record verification over four reports | if a verification grows past one window, partition it by relation, one verdict per subject, which the layout's verifies already permits |
| artifacts need lineage at the source and lineage | a source change should surface its downstream targets | reruns on upstream change | the engine already re-readies downstream jobs from the relations; the driver's report could list, for a corrected member, which verdicts lapsed, which the status command could show |
| acceptance authority needs no separate repair session | a worker can run the same check on its draft and repair in session; acceptance stays with the orchestrator | the worker's self-check | applied in form; the note adds that worker-local runs are unrecorded, so attempts count submissions, not repair effort, and that a repeatable check invites shrinking the draft to pass, so blockers must state purpose |
Cautions
- Correlated verifiers add little. Error correction works above chance oracles with decorrelated checks: a verifier on the same model reading the same members as the producer is correlated; retries cannot amplify a weak verdict oracle. Topology, isolation and verification: a verifier that reads the producer's answers loses isolation.
- Synthesis is not error correction. Merging propagates errors; the synthesis needs its own verification, which it has, and must stay within the cited records.
- Coverage is a predicate, not verification. Codification: a passing code check shows its predicate holds; the reports and docs should not call coverage "verified" beyond that. Semantic work can be relocated, not eliminated.
- A declared rule counts only with a consumer. Behavioral authority and operative change: audit that every layout declaration has a consumer, and that a
declinedanswer and a Limits entry are read by a later job or gate; otherwise they are inert records. - Volume is not effort. Cheap generation breaks text volume as an effort signal: long reports hand the verifier the unperformed checking; bound what each verifier must check.
- Workers reading untrusted sources. Privilege quarantine: a worker that reads the analysed system's code holds no authority beyond its output files, which the hand-out gives it; keep it so.
- Skill discovery re-fires in every worker. Skill discovery: prompt text that echoes a skill's description makes the worker load it; the prompt section already tells workers the authoring skills do not apply, which is the right shape.
- Scaffolding is per model. Scaling absorbs scaffolding at fixed difficulty: attempts and hand-holding tuned for one model should be revisited per model, not frozen.
- Final success does not show path health. Final task success: an acceptable member by an off-plan route passes; record or check the path, not only the output. Selecting an output fixes bytes, not interpretation.
- Derived copies must be checked or absent. A derived copy of recomputable truth: any hand-written copy of a layout fact in an instruction is the drift the derived lines remove; keep none.
- Blockers must be actionable for their addressee. Actionable methodology: a blocker aimed outside the producer's write scope needs a route, which
declinedpartly gives. - Procedural missions are relaxing candidates. Operational signals of a relaxing candidate: a mission that prescribes steps rather than outcomes is brittle; the shrink to establish-and-challenge is the right direction.
Applied already
The sweep found the design in step with: frontloading and model-resolved indirection (slots as lines, derived types, layout facts); generate instructions at build time (the rendered prompt section); fix what the executor can't determine and intent-framed delegation (missions); scheduler and LLM separation, LLM-mediated schedulers degrade and runtime structure determines governance surfaces (the engine); verification needs a typed target and document types should be verifiable (the layout); deterministic validation should be a script and semantic review catches what structure misses (checks under verdicts); criteria edits invalidate verdicts (judgments pin their criteria); instructions are typed callables (reads and types); history has one chance to become checkable and capture at the decision surface (answers with reasons); context is composed without scoping and vocabulary collisions prevented at write time (fresh workers, ADR 116); moving the interpretation boundary requires coverage (relations derived in code, judged by verifiers).