LLM-Wiki-v3 report verification
Type: agentic-system-analyses/types/agentic-system-verification.md
Verification
The pinned runtime, memory, epistemic, workflow and reconciliation members support a bounded, code-grounded account. I read all supplied members, their types and the record contract. The structural report contains no findings; the four correction-answer files are empty. There are no correction arguments to accept or decline. I independently inspected the implementation rather than relying on the analysts' validation statements. Source HEAD equals the frozen revision and its working tree is clean. No target code, tests, providers or examples were executed.
Source and record checks
The boundary remains appropriate: the shipped CLI, seed driver, retained store, retrieval and evaluation belong to one memory-system loop, not an enclosing agent runtime. The top-level tracked tree agrees with its coverage table. Additional inspection remained inside SRC-1. Source anchors below denote implementation unless identified as doctrine or a retained artifact. No new source identity or evidence layer is required.
I checked every declaration and annotation for referent, supporting source passage, scope, evidence strength and consumer consequences. Runtime's model components distinguish remote extraction from local embedding/reranking, configured names from frozen artifacts, and inference from parameter training. Their call sites support those distinctions (SRC-1, src/wiki/config.py, src/wiki/extract.py, src/wiki/retrieve/dense.py, src/wiki/retrieve/rerank.py). The route and policy findings retain implementation versus operation statuses; none claims that code inspection establishes successful operation, semantic fidelity or benefit. See runtime records.
Memory's objects distinguish source bytes, access metadata, prose, routing fields, provenance, diagnostics, lexical/graph/vector derivatives and reporting artifacts. I checked their writers and actual later readers, including metadata-triggered raw-store reuse, source-ID-only query provenance and bootstrap's body-only delivery. Those support the stated storage, lineage, authority and invalidation distinctions, not a generalized learning claim (SRC-1, src/wiki/ingest/hashing.py, src/wiki/ingest/pipeline.py, src/wiki/retrieve/store.py, src/wiki/retrieve/query.py, src/wiki/cli/__init__.py, src/wiki/audit.py, src/wiki/eval/harness.py). See memory records and read-back.
The epistemic objects and ledger preserve the difference between candidate assertions, checks, operational disposition, retention, ranking and acceptance for truth. I traced every declared subroute to its implementation and checked its architectural assessment against its separate operation status. Extraction and generated-label semantics remain indeterminate rather than inferred faithful or ampliative. Numerical metric derivations are scoped to their input conventions, not treated as domain knowledge. The policy claims are attributable to source doctrine and are compared with executable behavior rather than adopted as guarantees. See epistemic records and bounded conclusion.
Consequential behavior and alternate paths
- Extraction and eligibility. The pipeline actually supplies root instructions and a fresh chunk to the proposal tool call. Quote location precedes structural and marked-claim checking; the first nonempty all-valid set wins, otherwise the last attempt supplies staging candidates. First-ID deduplication can discard a later valid duplicate. The stronger attempt does not receive a critique or merge the earlier valid subset. This supports both RT-RTE-source-ingest and the workflow account, including its partial-staging and completion qualifications (SRC-1,
src/wiki/extract.py,src/wiki/ingest/pipeline.py,src/wiki/ingest/types.py). Chunk accumulation and overlap support approximate, not strict, input budgeting (SRC-1,src/wiki/ingest/chunk.py). - Provenance and measurement. I checked the loops and their domains, not only their names. Both structural citation checking and lexical checking enumerate existing markers. The schema loader merges frontmatter contracts; the base schema does not impose prose citation coverage. Thus RT-CLM-provenance-policy correctly distinguishes all-claim doctrine from partial enforcement. The no-marker maximum-score deduction in EPI-RTE-ingest-measurement follows from the actual early return. It does not imply that an uncited page with no sources would be included by the CLI evaluator, which separately skips pages lacking sources. The report retains that selection condition (SRC-1,
src/wiki/validate.py,src/wiki/schema_loader.py,schema/v1/_base.schema.yaml,src/wiki/ingest/spancheck.py,src/wiki/eval/ingest_eval.py,src/wiki/cli/__init__.py). - Admission and history. I traced interactive edit/accept, callable acceptance, installed-hook dispatch, supported
--no-commitand seed writes. Moving occurs before optional Git operations; acceptance does not invoke schema/span checks. Seed validates then writes directly. These are supported alternative paths, not bypasses inferred from unchecked types. Same-name replacement and the lack of reciprocal supersession updates justify the qualified history account. Retained manifest warnings are not automatically delivered to review (SRC-1,src/wiki/review.py,src/wiki/cli/__init__.py,scripts/pre-commit,scripts/seed_corpus.py,src/wiki/ingest/diff.py). See RT-RTE-page-review, RT-RTE-curated-seed, RT-CLM-history-policy and MEM-OBJ-ingest-manifest. - Withdrawal and freshness. Default current-page loading is real, but does not constrain every returned fused ID. I traced BM25, graph and dense production, graph seed selection and traversal, fusion, active reranking and fallback. Ordinary supported query after page changes without rebuild can use stale indexes; graph traversal also lacks target-existence filtering. Active reranking filters its candidates, while import failure can return to ordinary fusion. The records correctly avoid either universal exclusion or universal failure claims. Independent component replacement is not a shared transaction (SRC-1,
src/wiki/retrieve/store.py,src/wiki/retrieve/rebuild.py,src/wiki/retrieve/bm25.py,src/wiki/retrieve/graph.py,src/wiki/retrieve/dense.py,src/wiki/retrieve/fusion.py,src/wiki/retrieve/query.py,src/wiki/retrieve/rerank.py). See RT-RTE-index-query and MEM-OBJ-graph. - Evaluation and later model use. The direct baseline loader, component adapters and ingest evaluator have different selection paths. Their metrics consume expected IDs or source slices, not an independent truth oracle. Bootstrap sends retained bodies to the model, assigns labels from filenames and writes every nonempty response line. Its Path option can name the active dataset; the review reminder does not enforce a separate merge. These conclusions follow through producer and consumer, including supported input reachability (SRC-1,
src/wiki/eval/baseline_retriever.py,src/wiki/eval/harness.py,src/wiki/eval/metrics.py,src/wiki/cli/__init__.py). See RT-RTE-eval-report, RT-RTE-query-bootstrap and EPI-RTE-query-bootstrap. The saved baseline Markdown is an attributed report, not a reproduced experiment (SRC-1,evals/baselines/baseline-20260529T144104Z.md).
Relations, conflicts and omission disposition
I checked every declared Part of: relation. The import, extraction, quote, selection and pending-retention functions fit inside the ingest route; review disposition and retained move fit inside review; query selection fits inside rebuild/query; measurement functions and their output fit inside evaluation; generated-query transformation fits inside bootstrap. Shared schema and span evaluators correctly remain outside exclusive containment because they have multiple consumers. An output object's route parent is explained rather than mistaken for object identity.
The reconciliation makes two supported identity amendments: page bodies and evaluation-query objects. Neither upgrades evidence or changes a finding. Its remaining raw/meta, page-metadata and reporting containers overlap without being identical; retaining them rather than forcing split supersession is consistent with their valid scoped findings. I checked all memory integration issues and all cross-report comparisons. Baseline-versus-index retrieval, warning delivery, provenance loss, seed admission, retry replacement and implementation-versus-operation terminology are compatible distinctions. No unresolved disagreement or missing material part requiring correction was identified.
Omission questions were selected from the frozen tree and writer/reader dependencies: whether another shipped producer invalidates human-admission claims; whether another corpus consumer invalidates index-only or current-page claims; whether logs, manifests or evaluation outputs feed an automatic policy/promotion loop; and whether intended MCP/compile behavior supplies a hidden answering consumer. Searches across SRC-1 src/, scripts/, README.md, AGENTS.md and Makefile, followed by the call-site reads above, support the reports' path-specific answers. In particular, the seed driver and baseline loader are included, bootstrap is a distinct later model consumer, and CLI/Makefile dispatch does not implement the advertised future answering service. Audit/report consumer searches support scoped non-updater findings, not a negative about arbitrary external tools. The standalone gold-ID coverage/precision helper in src/wiki/eval/ingest_eval.py has no shipped caller in those searches and does not alter the account of active evaluator authority.
Further omission expansion stopped because these questions have supported dispositions and no identified material challenge remains unresolved. Auditing every seed specification, corpus assertion, optional dependency or test body would not establish the missing observed-use or causal links and is unnecessary for the emitted architectural claims. Such content was not treated as evidenced absence. The reports already withhold corpus-wide truth/provenance replay, exact external model fixity, actual human review, downstream answering use and improvement conclusions. Those faithfully scoped uncertainties remain beside the claims they bound; they require no additional verifier limit or correction.
Blockers
none
Limits
none