Workshop: agent-note-improvement

Goal: find instructions that help agents improve existing notes, either by moving a weak version toward a later accepted version or by accurately marking why the weak version should not stand as-is.

Starting question

Can an agent, given an existing note that has already passed through automatic improvement mechanisms, identify the parts that still make it weak?

The target behavior is not generic polishing. A useful instruction should do one or more of:

  • find the note's strongest load-bearing point;
  • identify speculative or low-yield expansions that make that point weaker;
  • distinguish weak-but-plausible material from material that is simply false;
  • recommend removal, compression, or isolation without erasing the central claim;
  • explain what evidence or argument would be needed before a speculative section deserves to stay.

Draft review bundles

Combined instruction

Promoted 2026-07-07 to kb/instructions/run-full-improvement-pass-on-note.md, alongside its compression-bundle runner at kb/instructions/run-compression-bundle-on-note.md. Both dropped their kb/work/-relative dependencies (the case-comparison pointer, the workshop-local gate paths) as part of the move; see those files for the current procedure. Sequences the compression bundle, critique-note, composition-friction-gate, and the production semantic bundle and cp-skill-connect over one note, then reconciles their findings into a single editorial packet. Drafted from the interim comparisons below (case 01's table, case 02's critique-note vs. prune-weak-expansions split, case 03's compression-bundle-beats-single-gate result); run end-to-end for the first time in case 04, with composition-friction-gate added after case 04 and exercised for the first time in case 05. Promoted on 5 clean runs (cases 04–08, the last three via kb/log.md-adjacent library-note passes) with the semantic-bundle bar met but the two-methods-conflict-on-the-same-passage bar still unmet — see "What would close this workshop" below.

First case

Case 01 uses kb/notes/llm-generation-relaxes-goals-where-human-writing-stalls.md.

  • baseline-e242c975 — historical version from e242c975a2542b88d43b5d609ebbca27fd3bf3cd, after substantial automatic improvement but still weak.
  • current-2026-06-16 — current accepted version, copied on 2026-06-16.
  • case notes — target delta and experiment log.

Second case

Case 02 uses kb/notes/prose-has-no-dereference-reinforce-facts-at-point-of-use.md.

Third case

Case 03 uses kb/notes/adversarial-loop-can-reconstruct-the-writing-is-thinking-filter.md.

Fourth case

Case 04 uses kb/notes/a-derived-copy-of-recomputable-truth-must-be-checked-or-absent.md — the first live run of run-full-improvement-pass-on-note, applied directly to a current library note rather than a frozen baseline/accepted-delta pair, so there is no separate "accepted delta" to compare against; the edit was made in place.

  • Reports: compression-bundle-review; critique-note report at kb/reports/critique/a-derived-copy-of-recomputable-truth-must-be-checked-or-absent.critique.md; connect report at kb/reports/connect/notes/a-derived-copy-of-recomputable-truth-must-be-checked-or-absent.connect.md (semantic bundle skipped — exploratory pass, not a promotion decision).
  • Synthesis packet: full-pass-report.

Result: clean end-to-end run, no reconciliation failures.

  • Compression and critique-note found genuinely different things, matching the instruction's ordering rationale: compression flagged two passages that had outgrown their argumentative job ("What enforcement buys," half of "Where the rule applies"), while critique-note surfaced a real unaddressed gap — the validator-correctness regress (a validator is itself a hand-maintained artifact making an unchecked claim). Reconciliation kept both: cut the compression-flagged passages and added a fourth precondition addressing the critique, netting out close to length-neutral.
  • Connect added three missing reciprocal links and caught one unauthorized footer label (complements, not in kb/notes/COLLECTION.md's label set) — a small maintenance find none of the other three methods could have caught.
  • The step 8 flow/coherence pass made only two small wording fixes and reintroduced nothing, confirming the "copyedit only, do not reopen content decisions" boundary held in practice, not just on paper.

Takeaway: this is the first case where all four methods' outputs were genuinely complementary rather than overlapping or conflicting — plausibly because this note (dense, many links, single-thesis) sits squarely in compression's and critique-note's separate strike zones rather than needing one to arbitrate the other. A future case should stress-test the reconciliation rules on a note where compression and critique actually disagree about the same passage, which did not happen here.

Fifth case

Case 05 uses kb/notes/structure-inference-needs-capture-at-the-decision-surface.md — the first run of run-full-improvement-pass-on-note including the new, experimental composition-friction-gate step. Applied directly to a current library note; no frozen baseline/accepted-delta pair.

  • Reports: compression-bundle-review; critique-note report at kb/reports/critique/structure-inference-needs-capture-at-the-decision-surface.critique.md; composition-friction-gate report at kb/reports/friction/structure-inference-needs-capture-at-the-decision-surface.friction.md; connect report at kb/reports/connect/notes/structure-inference-needs-capture-at-the-decision-surface.connect.md (semantic bundle skipped).
  • Synthesis packet: full-pass-report.

Result: the first case with real cross-method corroboration, and the first real test of the friction-gate's "carry unresolved" rule.

  • critique-note and composition-friction-gate independently converged on the same sentence — a categorical overclaim ("changes which world models are learnable at all") — from two different mechanisms: critique-note via an external counterexample (decision/process mining defeats the claim's "only if" at the population level), friction-gate via an internal tension with the note's own Open Questions (a continuum admission undercutting a categorical claim). Independent corroboration across orthogonal methods is exactly the signal error-correction-works-above-chance-oracles-with-decorrelated-checks predicts, and it's what justified turning the finding into an actual edit — a real scope-narrowing (rationale-bearing structure vs. population-level regularity mining), not just a caveat.
  • Friction-gate surfaced four other thinnest-joint findings with no corroboration from any other method. All four went into the packet's "Routed attention" section and were not converted into edits, per the instruction's rule — including at least one (the "nearly free" claim) that would have been an easy, tempting fix. The rule held under real pressure, not just on paper.
  • Compression again found real cuts (two paragraphs duplicating footer content) in locations untouched by critique-note or friction-gate — the three report-only methods stayed non-overlapping in what they flagged.

Takeaway: this case exercised the two things case 04 didn't — genuine cross-method corroboration forcing a real (not just additive) edit, and the friction-gate's no-self-grading rule under a live temptation to "just fix" an easy-looking finding. It still didn't produce the originally-flagged test (compression and critique-note disagreeing about the same passage); that specific scenario may be rarer than expected, and corroboration-vs-isolation across methods looks like the more informative axis to track going forward.

Experiment pattern

For each instruction under test:

  1. Freeze the weak baseline and later accepted version.
  2. Run the instruction against the baseline in a fresh agent.
  3. Compare the output against the accepted delta.
  4. Record whether the instruction found the weakness, proposed a direction compatible with the accepted revision, or missed the target.

What would close this workshop

The workshop closes when it produces one of:

  • a reusable note-improvement instruction — done, 2026-07-07: kb/instructions/run-full-improvement-pass-on-note.md. Case 04 was a clean, non-overlapping run; case 05 added composition-friction-gate, produced real cross-method corroboration (critique-note + friction-gate on the same sentence), and held the friction-gate's "carry unresolved" rule under real pressure. Cases 06–08 (the three-notes pass, kb/log.md/commit 6c754cf9) each ran the full pipeline including the semantic bundle, satisfying that bar. Promoted on that evidence even though the originally-flagged bar — two methods actually conflicting (not just corroborating or staying isolated) on the same passage — still hasn't occurred across any of the 5 documented runs; treat that as an open question for the instruction's own future use, not a blocker met retroactively;
  • a critique or review-gate revision that reliably catches weak speculative expansions;
  • a negative result explaining why the tested instruction family does not recover the accepted edits;
  • a methodology note about when agent note-improvement should be subtractive rather than additive.

Relevant local context

Complete file listing (generated at build time)