Generator retrodiction run

Date: 2026-07-29 (all three variants) Status: three variants executed — palette-anchored, palette-free, and family-invention; results adjudicated by the maintainer session, not independently replicated. Tests: the "generator retrodiction" item in competing link models — can stage 1 of the candidate generator (seed a collection's link vocabulary from what its artifacts denote) reproduce the live authorized vocabularies from contract descriptions alone?

Protocol

For each of the nine contracts (notes, reference, instructions, agent-memory-systems, agentic-systems, sources, articles, types, work; the dialectical-sample fixture excluded):

1. Extraction (blind split). An extractor agent split each COLLECTION.md into a blind input — purpose, quality goal, what artifacts denote, title conventions, destination names, search guidance with label tokens redacted — and a ground-truth table of authorized labels per destination, copied verbatim. A deterministic grep over the blind files then checked for residual label tokens; two soft leaks were manually neutralized ("link a Commonplace procedure" in the agentic-systems guidance, verb "grounds itself" in work).
2. Prediction. A fresh predictor agent per collection received only: the stage-1 procedure (endpoint-kind classification, family-licensing table without corpus witnesses, per-pairing selection), the collection's blind input, and the shared label catalogue as palette. It was forbidden to read anything else and predicted authorized labels per destination.
3. Scoring. A scorer agent compared predictions to ground truth per (destination, label) pair — exact hits, misses, false alarms — with part-of/contains-style pairs unified. Full detail in the session scratchpad (retro/scoring.md); this file preserves what matters.

Extraction, prediction, and scoring ran on Sonnet; adjudication below is the maintainer session's.

Known blindness limits, recorded up front. The catalogue itself leaks three things: the register-of-origin grouping (hints family-to-register assignment), rests-on's parenthetical direction note, and operationalized-from's sentence naming its currently authorized notes→instructions pairing — the one predicted operationalized-from hit is therefore discounted. The palette also still contains the pre-migration labels (grounds, mechanism), so predictors could hit them exactly; no prediction ever coined the successor labels (premised-on, explained-by, operates-through), which says the palette anchors label choice completely.

Results

collection	truth labels	exact hits	misses	false alarms
notes	28	14	14	6
reference	25	11	14	2
instructions	7	4	3	1
agent-memory-systems	18	8	10	3
agentic-systems	15	7	8	1
sources	16	9	7	2
articles	0	0	0	0
types	6	2	4	3
work	72*	17	55	18

* work is a weak-ground-truth case twice over: its contract marks labels as suggestions, and the scorer expanded its any-destination working-state row across eight destinations, inflating every count. It is excluded from the aggregate below, as is the degenerate-but-correct articles.

Aggregate over the seven contractual collections: 115 truth rows, 55 exact hits (48%), 60 misses, 18 false alarms.

The misses decompose almost entirely into two systematic classes:

• 30 of 60 are see-also. Predictors treated the escape hatch as dominated by any specific label; the contracts authorize it in parallel almost everywhere.
• 18 of 60 are lineage labels (derived-from, abstracted-from, plus one adapted-from-adjacent case). Predictors either rejected lineage outbound from theory/description collections or picked the wrong member (agent-memory-systems → sources: predicted abstracted-from from the contract's prose word "abstracted"; truth wants derived-from).
• The remaining 12 are scattered: five evidenced-by placements, two control-flow labels (composition, precondition — family right, selection incomplete), rests-on from sources, two compares-with, one defined-in, one depends-on.

Removing the two systematic classes: the residual surface scores 50/62 = 81% exact.

What landed

• Intra-collection families were recovered almost perfectly blind. notes→notes: 8/8 (the full inferential set, including both pre-migration labels). reference→reference: 4/4 (the full structural/versioning set). sources→sources: 2/2. instructions→instructions: 2/4 within the right family.
• Cross-kind signature relations were repeatedly right, including the genuinely non-obvious ones: rests-on predicted for every X→notes theory-dependence pairing that authorizes it (reference, both external-systems collections, types, instructions); operates-on for instructions→reference; procedure for description→instructions; and — notably — the direction of the evidence pair: is-evidence-for predicted from the observation-side collections (sources, both external-systems collections) toward notes, evidenced-by from the claim side.
• articles: full structural match. The predictor concluded "no labelled surface at all, unlabelled external citations only" — correct, on a contract a naive generator would have over-populated.
• types' destination prohibitions were respected (only notes and reference predicted).

What failed, and what each failure teaches

1. see-also is a policy constant, not endpoint-kind information. Its parallel authorization is a KB-wide convention. A seed can carry it as a constant; its absence from predictions says nothing about the endpoint-kind hypothesis.
2. Lineage is a second, independent vocabulary dimension. The lineage labels are indexed by production history and maintenance regime (recheck vs re-derive vs re-examine), not by what the endpoints denote — the exact boundary the catalogue's lineage semantics and the open lineage-mechanisms workshop adjudicate by hand. Endpoint kinds cannot generate them, and the one directional confusion the predictors made (abstracted-from vs derived-from) is precisely the distinction whose truth condition is a maintenance regime. Either the seed gains a third input — the collection's production relations, which contracts do describe (ingest workflows, snapshot-to-analysis paths) — or lineage stays harvest-only.
3. types→reference is the strongest single counterexample. Truth wants depends-on, evidenced-by, see-also; the predictor coined implements, part-of, operates-on — a family-level flip, reading the type-contract→system edge as structural realization where the contract asserts dependence. Endpoint kinds license several families for a pairing and underdetermine which one the collection commits to; the authority direction between a contract and the system it constrains is a decision, not a consequence of artifact kinds. (depends-on is also absent from the shared catalogue — a palette gap.)
4. Truth is consistently narrower than family licensing. Nearly all false alarms are family-plausible over-extensions (compares-with into reference, is-evidence-for into destinations that only grant see-also, control-flow labels for work→instructions). The live contracts authorize a curated subset of what endpoint kinds license.
5. work's working-state labels (draws-on, tests, depends-on, produces) were never predicted anywhere. They encode workshop process state — a dimension neither endpoint kinds nor the palette carries. Consistent with the workshop layer being a different kind of surface, and with its own contract calling the labels suggestions.

Verdict

The test's pre-registered criterion was: strong overlap plus explicable residue supports the generator; a collection whose vocabulary doesn't track its endpoint kinds refutes it.

The result supports the endpoint-kind seed at the family level and refutes stage 1 as a label generator. Family structure was recovered blind — intra-collection families near-perfectly, cross-kind signatures including evidence direction repeatedly — and no collection's vocabulary failed to track its endpoint kinds. But 48% exact overlap means stage 1 alone cannot write a contract's label table: it cannot see policy constants (see-also), cannot see the lineage dimension at all, underdetermines family choice where authority direction is contested (types), and systematically over-generates relative to curated authorization.

Sharpened generator claim for the workshop: seed = endpoint kinds + production relations + policy constants, emitting a superset per destination pairing; harvest prunes and decides. The dimensions stage 1 provably cannot see — maintenance regime, production history — are the same ones the migration program keeps adjudicating by hand, which is convergent evidence that they are real, independent semantic dimensions of the vocabulary rather than classification noise. This strengthens the two-stage design in competing link models and weakens any hope that a contract alone could ever fully generate its vocabulary.

Palette-free variant

The first run could not distinguish "predictors select well from a palette" from "endpoint kinds generate families." The variant removed the catalogue entirely: fresh predictors received the same blind inputs plus a family inventory with neutral descriptions only (no label identifiers, no per-label reader-need lines, no direction annotations, no register grouping) and had to coin every identifier themselves, tagging each coinage with a family code and a source <label> target assertion. A scorer then matched coined relations to truth at two levels — family assignment per (destination, family) pair, and relation-level gist-plus-direction per truth label — using a fixed truth-label→family mapping.

Honest scope note: the variant withholds names and selection, not the family inventory itself — the inventory is harvested from this corpus, so the variant tests whether the palette was carrying the semantics, not whether the families could be invented from nothing.

Results (seven contractual collections, as before)

• Family level: 88 truth (destination, family) pairs; 41 hits (47%), 16 false alarms. One judgment call dominates: the agent-memory-systems predictor coined a single weak-navigation catch-all at destination any; expanding it adds 7 NAV hits (unexpanded: 34 hits, 39%). On the non-NAV surface the result is expansion-independent: 32/58 = 55%.
• Relation level: 53 of 115 truth labels matched a coined relation in family, gist, and direction (46%; 40% unexpanded) — statistically indistinguishable from the palette run's 48% exact, despite every identifier being invented blind.
• The non-NAV family misses are omissions, not semantic errors. Of 26: 15 are EVI and 8 are LIN edges left un-enumerated at pairings where the same predictor authored the same family correctly elsewhere. Only three are genuine mis-assignments: types→reference (again — see below) and reference substituting structural comparison for the lineage/evidence families toward the two external-systems collections.

What the variant establishes

1. The semantics are generated, not selected. Evidence direction was never once misdirected: every observation-side collection coined an observation-side relation (counts-as-case-for, bears-on, corroborates), every claim-side collection a claim-side one (corroborated-by, supported-by, informed-by). Theory-dependence was coined independently by six collections at exactly the X→notes pairing that authorizes rests-on (assumes, grounded-in, presupposes-claim, assumes-framework — and, in agentic-systems, the literal string rests-on). The reference predictor rebuilt the entire intra-reference structural family gist-for-gist (aggregates, realizes, supersedes, parallels).
2. Convergent reinvention. Six coined identifiers are character-identical to live labels (rests-on, supersedes, invokes, extends, defined-in, evidenced-by) and several more are near-identical (acts-on≈operates-on, conflicts-with≈contradicts, operationalized-into≈operationalized-from). Blind re-derivation converging on the live vocabulary's names is stronger evidence for the generator than palette selection could ever be: the labels behave like attractors of the endpoint-kind + reader-need computation, not like arbitrary conventions.
3. The palette contributes recall, not precision. With the catalogue gone, per-pairing enumeration thinned out (the 23 EVI/LIN omissions; reference predicted no evidential edge anywhere versus five correct ones in run 1) and the NAV blackout became total — eight of nine predictors explicitly reasoned the escape hatch away. The catalogue's real function in run 1 was a completeness checklist and a name supply; the semantic content survived its removal intact.
4. Lineage sharpens rather than repeats. Unlike run 1's predictors, the palette-free ones did author lineage from notes and reference — the family is visible in the contracts after all — but three collections independently collapsed the recomputable-copy and generalization regimes into one identifier that only fits generalization (generalized-from, condensed-from read as regime-c). The family is contract-derivable; the three-regime maintenance distinction is not. This narrows run 1's lesson 2 to its true core: what endpoint kinds cannot see is precisely the maintenance regime, not lineage as such.
5. types→reference fails a third way. Truth: dependence. Run 1: structural realization. This run: a cross-kind pointer (governed-by). Three tries, three different families — this pairing is genuinely underdetermined by endpoint kinds, and the authority direction between a type contract and the described system is a decision the contract makes, not a consequence of what the artifacts are. It is now the workshop's best worked boundary case.
6. articles regressed mildly: two hedged coinages against truth's zero labels, versus run 1's exact zero-for-zero. Weak signal, worth having on record because it cuts against the generator on the one degenerate contract.

Family-invention variant

The palette-free variant still supplied the family inventory, leaving open whether the families themselves were smuggled in. The third variant removed it: fresh predictors received only the endpoint-kind classification steps, the link grammar, and the two-consumer registration framing — with one addition, each coined relation had to state its revision consequence (what the source reconsiders when the target changes) alongside its assertion and reader need. Predictors derived their own relation categories from first principles and coined all identifiers. The scorer first classified every coined relation into the nine harvested families (or NOVEL) from its assertion/reader-need/revision-consequence text, then scored as before.

Results

• Family census — the variant's headline question: 9/9. Every harvested family (inferential, structural, evidential, theory-dependence, control-flow, procedure-pointers, definition, lineage, weak navigation) was independently re-derived and correctly placed at least once, the first variant to achieve full coverage. Most families recur across 3+ collections; control-flow appears once (only one collection is procedure-kind) and weak navigation once — but that once is clean: the sources predictor derived a cites→external escape hatch with an explicit no-dependency clause, unprompted.
• Coverage declined again: 48 of 116 mappable truth labels matched (41%), versus 46% (palette-free) and 48% (palette-anchored). The recurring omissions are the same three: the see-also policy, derived-from regime-a into evidential collections, and per-pairing enumeration thinning. (Scorer baselines drifted a few rows between variants — the same counts-drift phenomenon the corpus reviews documented — so cross-variant percentages are indicative, not precise.)
• types→reference resolved on the third try — correctly. Truth: dependence. Palette run: structural. Palette-free: cross-kind pointer. This run: depends-on triangulated three independent ways (conforms-to, is-enforced-by, is-consumed-by), each carrying a maintenance clause. The one substantive procedural difference from the previous variant is the required revision-consequence line — suggestive that making the maintenance dimension explicit is what disambiguates contested pairings. (Corrected by the A/B below: the revision-consequence hypothesis did not survive its controlled test, and the "contested pairing" itself dissolves under k-sampling.)
• Lineage regimes improved rather than collapsed: the notes predictor uniquely coined two distinct regimes correctly (is-derived-from→sources, regime-a; operationalized-into→instructions, regime-c-narrow) while regime-b went untouched; other collections still anchor one regime and miss the rest. Regime-a into the external-systems collections remains the most systematically missed lineage gist across all three variants.
• Evidence direction: three variants, zero flips. Every failure was omission or wrong-family substitution; no predictor in any variant ever reversed an evidential arrow. Direction is fully determined by endpoint kinds — this is now the single best-established generator claim.
• One credible NOVEL relation emerged: the reference predictor derived elaborated-by — summary-versus-full-account detail routing between two reference documents — which fits none of the harvested structural sub-gists. First instance of the generator running forward: proposing a possibly real gap in the live vocabulary rather than reproducing it. Worth a corpus check (are there reference→reference edges doing this work today under part-of or see-also?).
• Stable reproductions across all three variants: articles returned to perfect zero-for-zero abstention; the work predictor invented an execution-control relation into kb/instructions/ for the third time (either a genuine contract gap or a case where the workshop layer's sink-role policy deliberately overrides endpoint logic); and both system-review predictors again reasoned a strong structural comparison relation into pairings where the contract deliberately keeps only a weak link — a reminder that contracts sometimes under-commit on purpose, which no generator can see.

Revision-consequence A/B

The family-invention run suggested that requiring a revision consequence per relation was what fixed the contested types→reference pairing. The controlled test: ten fresh predictors on the types blind input (which contains no maintenance language — checked), five per arm. Arm A used the family-invention procedure verbatim (revision consequence required per relation); Arm B an otherwise-identical procedure with every maintenance/revision mention stripped, including the second consumer in the registration framing. A classifier then sorted every coined kb/reference/-destination relation into families from its assertion text, with verbatim deciding phrases for audit; it was not told the hypothesis or the arm meaning.

Result: the hypothesis is refuted. The dependence family appeared at types→reference in 10 of 10 runs — five per arm. The maintenance clause is not what disambiguates the pairing; endpoint-kind reasoning alone reliably produces the correct family. The kb/notes/ control was equally stable: dependence + definition in all ten runs, matching truth.

The bonus discovery reinterprets the earlier "contested pairing" story. Every run coined a stable portfolio at types→reference: a decision-dependence relation (DEP — decided-by, conforms-to, mandated-by…), an enforcement relation (classified STR — enforced-by, is-validated-by…), and usually a downstream-consumer relation (XPT — is-consumed-by, operates-within…). The "three tries, three families" pattern across the single-run variants was therefore portfolio sampling, not family confusion: single runs surfaced different members of the same stable portfolio. (This corrects the family-invention section's framing above; the correction is left in place deliberately.) The methodological lesson generalizes: single-run family assignments are unreliable — k-sampling changed the conclusion.

Two further stable facts across all ten runs: no run produced an evidential relation at either destination (truth's evidenced-by→reference stayed invisible — consistent with evidential-omission being the dominant systematic miss in every variant), and no run produced weak navigation (the policy constant again). And the portfolio surfaces two new forward-generator candidates: is-enforced-by and is-consumed-by at types→reference are coined by nearly every run, carry articulable reader needs (which validator checks this contract; what breaks if it changes), and are authorized by nothing — either deliberate under-commitment or genuine vocabulary gaps, joining elaborated-by in the corpus-check queue.

One weak secondary signal, recorded at low confidence: the consumer relation appeared as XPT in 5/5 Arm B runs but only 2/5 Arm A runs — in Arm A it was sometimes absorbed into a dependence reading via its revision clause (A3's consumed-in was classified DEP precisely because its revision consequence asserted forward dependence). If real, the maintenance clause shifts classification pressure at family boundaries rather than family choice; k=5 cannot establish this.

Corpus check of the forward-generated candidates

The three relations the generator coined repeatedly without authorization were checked against the live corpus (two read-only sweeps, 2026-07-29). All three survive: each names a real, recurring relationship the current vocabulary either forces into weaker labels or loses to unlabeled prose.

• elaborated-by (source covers X briefly; target is the full account): at least 15 independent instances across 10+ reference documents, clustering exactly where predicted — the entry-point and overview docs (README.md, lib-modules.md, README-REVIEW-SYSTEM.md, agent-memory-coverage.md's whole implementation-references footer). The relationship is currently split three ways, none clean: forced into contains/part-of (9 instances, with contains used backwards — the target holds the detail), left as bare inline "for the full X, see…" prose with no footer edge (9 instances), and one edge whose context phrase literally says "summarizes" filed under evidenced-by. The one nearby genuine see-also is a parallel reading, not detail routing — confirming no existing label carries this sense.
• is-enforced-by (a shipped validator/subsystem mechanically checks this contract): recurs across the type specs — the validator's checks on type-spec.md, tag-readme.md's marks, note.md's structural validation, review-gate.md's whole subject matter. Where authored at all it is mislabeled: type-spec.md carries the enforcement relation once inline with no label and once as depends-on pointing at ADR 038, which describes the enforcement mechanism rather than a background decision. Most instances are entirely unauthored despite ready targets (validation-contract.md, README-REVIEW-SYSTEM.md, lib-modules.md); the reference side states the relationship in prose from its own direction.
• is-consumed-by (a shipped subsystem reads instances of this type): recurs — the conformance wrapper reads type-spec.md bodies as gate text, the ProperDocs hook reads index-typed frontmatter, the validator reads tag-readme marks. Authored exactly once, as a see-also whose own context phrase says "reads this file"; the index.md consumption relationship is entirely unauthored despite documentation-site.md existing as a target.

What this establishes: the generator has a working forward mode. Relations it coined repeatedly, with articulable reader needs and no authorization, turned out to identify real vocabulary gaps — the same seed that retrodicts the harvested vocabulary also predicts where it is incomplete. Caveats: the three were checked because they were the strongest repeat coinages (selection effect — answered by the control below), and the types surface is small (eight specs, five authored edges), so its two candidates recur at the "pattern across most specs" level rather than by edge count. Whether any of the three gets registered is a harvest-layer authorization decision for the consistency workshop and the owning collection contracts — this workshop only records the evidence.

Control check: the forward mode discriminates

The base-rate worry — maybe any plausible coinage finds corpus echoes — was tested against a relation expected to fail: the structural-analogue relation (compares-with/has-analogue-in) that both system-review predictors coined toward kb/reference/ in every variant, where both contracts authorize only see-also. The check was run with the same strict criteria, instructed to judge the evidence both ways.

It failed decisively — on all three criteria. Across roughly 1,280 "Commonplace" mentions in the two collections (most generated by the review type's mandatory "Comparison with Our System" section) and only three authored edges to kb/reference/ in total, exactly one arguable doc-targeted correspondence exists (thalo-type-comparison.md, one file, unlabeled). The sampled mentions (~90 sites, 22 files) are overwhelmingly diffuse whole-system contrasts naming generic Commonplace features with no specific reference document a link could target. And criterion (c) is contradicted outright: both COLLECTION.md files verbatim pre-declare "scan when a design element has a direct Commonplace analogue. Labels: see-also" — the exact reader need the candidate would serve, deliberately assigned to the weak label. Where a comparison does rise to formal correspondence, the corpus already routes it through kb/notes/ theory edges (rests-on, is-evidence-for in kb/agentic-systems/exo.md) rather than doc-to-doc.

Consequences: the forward mode discriminates — confirmed candidates had recurring, doc-targeted, mislabeled-or-missing instances; the control, coined just as insistently by the same predictors, has essentially none. Two findings ride along. First, this is the corpus's clearest demonstrated case of deliberate under-commitment earning its keep: the contract authors anticipated the analogue need and chose the escape hatch on purpose. Second, it explains why the seed over-fires here — the KB's actual practice expresses cross-system correspondence through shared theory claims ("both systems instantiate claim X," carried by authorized theory edges), a mediation-by-theory architecture choice that endpoint-kind reasoning cannot see. Both belong to the policy/harvest layer, exactly where the combined verdict already places everything the seed cannot generate.

Incidental finding for the consistency workshop: the reference→reference sweep's label census surfaced a long tail of live labels beyond the shared catalogue (decision ×11, foundation ×6, refines ×5, outcome ×4, amended-by ×3, and a dozen singletons) — mostly ADR-local vocabulary whose authorization status deserves its own check.

Combined verdict of the four runs

Stage 1 generates the semantic skeleton of a collection's vocabulary — the family inventory itself (9/9 re-derived), directions (never once flipped across ~40 predictor runs), gists, and to a striking degree the names — and generates none of the authorization table's remaining content: per-pairing completeness, policy constants like see-also, full lineage-regime coverage, evidential-edge enumeration, and deliberate under-commitments all live in the policy/harvest layer. Coverage falls monotonically as scaffolding is removed (48% → 46% → 41%) while semantic precision holds — the scaffolding buys recall, never semantics. The revision-consequence lead did not survive its controlled test: the maintenance axis is not needed for family precision, and the one "contested" pairing dissolved into a stable over-complete portfolio from which the contract selects. That sharpens the division one last time: the seed reliably emits a superset portfolio per pairing; authorization is the act of selection from it, and selection is exactly what the corpus record — not the contract description — determines.

Validity caveats

• One predictor per collection, one model (Sonnet), one run — no inter-predictor agreement measured; specific hit/miss rows may not be stable under resampling.
• The catalogue leaks listed under Protocol; the operationalized-from hit is discounted, the rests-on direction note may have helped its five hits.
• Ground truth includes pre-migration labels (grounds, mechanism) that the palette also contains; post-migration the palette-anchoring effect would change.
• Extraction blindness relied on agent judgment plus one grep; residual paraphrase-level leakage (contract prose describing a relation without naming it) cannot be fully excluded — the agent-memory-systems abstracted-from error shows predictors do read such prose.
• The scorer's any expansion for work was an interpretive choice; work numbers are indicative only.
• Palette-free variant specifically: the family inventory was still supplied (only names, reader-need lines, and annotations were withheld), so family invention was untested there; relation-level matching required semantic judgment by the scorer (gist and direction of coined assertions), which is softer than run 1's exact-string comparison; and the agent-memory-systems any-catch-all expansion is a single call worth ±7 hits, reported both ways above.
• Family-invention variant specifically: the scorer both classified coined relations into families and scored them — a single judgment chain with no independent check; the destination-role list in step 2 still names each collection's kind, so endpoint-kind classification of destinations was given, not derived; and the revision-consequence requirement changed alongside the inventory removal (the A/B subsequently isolated it and found it does not drive family choice).
• A/B specifically: one collection, k=5 per arm, one model; the classifier saw the A/B filenames (though not the hypothesis or arm meaning); and the header/status/date lines of the two theory files necessarily differed slightly beyond the manipulation. The 10/10 dependence result is robust to all of these; the weak XPT-absorption signal is not.

Next

• The prospective test remains the real one: seed the next genuinely new collection and count unpredicted harvested labels.
• Hand the three confirmed gap candidates to the harvest layer: elaborated-by (reference contract owner), is-enforced-by and is-consumed-by (types contract owner) now have corpus evidence dossiers above; registration, spelling, and inverse decisions belong to the consistency workshop and the owning contracts, not here. The elaborated-by migration surface is nontrivial (~18 edges/prose sites across 10+ docs); the types surface is 5 edges plus ~11 missing-edge sites.
• ~~Control the forward mode~~ — done (see the control-check subsection): the control failed as required on all three criteria; the forward mode discriminates real gaps from over-generation.
• The reference collection's ADR-local label tail (decision, foundation, refines, outcome, amended-by, singletons) needs an authorization-status check — consistency-workshop scope.
• The revision-consequence question is settled for family choice (A/B null) but not for family-boundary classification pressure (the weak XPT-absorption signal) — only worth pursuing if a downstream decision ever hangs on it.
• If reliability numbers are wanted for the main variants: k predictors per collection, agreement per (destination, label) — the A/B demonstrated single runs mislead at exactly this granularity.