LLM Repo Wiki report verification
Type: agentic-system-analyses/types/agentic-system-verification.md
Verification
Boundary, evidence and disposition
The reports support a bounded implementation account. I read all supplied members, their contracts, the empty correction-answer files and the structural report-check, which reports none. There is no structural failure to route and no previous semantic refusal to carry forward. I independently inspected the frozen source rather than adopting the analysts' validation statements as semantic verification. Git HEAD equals the reviewed boundary and the source worktree is clean. No target execution, model calls, tests or benchmarks were performed.
The boundary correctly includes the application, worker, instructions, benchmark callers, admission surfaces and retained-content consumers. The tracked tree and package entry points agree with that classification. The source register's implementation, doctrine/design and reported-operation layers remain distinct. No supplied database or raw execution artifact supports an upgrade to observed operation or causal benefit. External dependency implementations and host/provider behavior are not silently attributed to this repository.
I checked every declared record and its retained source passages, route status, consumer and evidence qualification; the workflow's annotations; the sole Part of: relation; the supersession and overlap dispositions; and the reports' potential disagreements. The following groups retain the consequential checks and their dispositions. Source anchors below are in SRC-1 at the reviewed commit. Existing record passages supply the load-bearing quotations; the checks do not add new analyst records.
Runtime and admission
- RT-RTE-repo-sync: traced add/manual-sync, poll and signed default-branch webhook inputs through worker registration, full/incremental file processing, filtering, secret scanning, per-file persistence, stale marking and author scheduling. The full and incremental branches both call preparation; incremental rejection can remove previously indexed content. The account correctly bounds this to detected secrets and inspected ingestion paths. It does not promise semantic freshness or atomic snapshot replacement. Sources:
src/ingest/repos.ts,src/ingest/poll.ts,src/ingest/webhook.ts,src/ingest/jobs.ts,src/ingest/sync.ts,src/ingest/prepare.ts. - RT-RTE-author-pages: read the complete author implementation, including initial, cross-repository extension, incremental and uncovered-change branches. Code owns retrieval; separate model calls propose plans and drafts. Both create and update use proposal mode. Grounding permits new hit-contained references or exact kept keys; the nonzero-citation test is not a sentence-support test. Evidence clipping and full metadata bounds are genuinely different. Idempotency, per-page failure, systemic-error propagation and between-call budgets support the bounded recovery account, not guaranteed completion. Sources:
src/agent/author.ts,src/agent/jobs.ts,src/agent/prompts.ts. - RT-RTE-revision-admission: traced UI actions through store locking, status and parent checks, direct apply-mode edits, rejection and accepted-body restoration. No human-authentication or factual-acceptance invariant follows from the actor label. Index enqueue occurs after persistence and catches failure. The workflow's bulk ordering and partial-result account matches
src/wiki/review.ts; its sync-to-author ordering matches the enqueue beforesyncedShaadvancement. These checks support the selected workflow, including its separate proposal, review and indexing endpoints. Sources:src/app/w/[wiki]/actions.ts,src/wiki/store.ts,src/wiki/review.ts,src/jobs/queue.ts,src/ingest/sync.ts. - RT-RTE-chat-turn: independently followed the supported apply-mode caller into tool execution and store creation. Creation does not appear in the approval map; update, move and delete do. This is a reachable shipped alternative to proposal review, not an inference from an unused API or TypeScript types. Message merging preserves stored tool arguments and accepts decisions on stored pending approvals. Actual signature/approval enforcement still depends on the external SDK. Search-first and cite-every-claim remain instructions; ordinary chat has no groundDraft call. Sources:
src/agent/chat.ts,src/agent/chat-store.ts,src/agent/tools.ts,src/agent/tool-names.ts,src/agent/models.ts.
The four model-component records—RT-CMP-chat-model, RT-CMP-author-model, RT-CMP-embedding-model, RT-CMP-rerank-model—correctly separate remote parameters from stored outputs/vectors. Configuration pins identifiers, not exact weights. Summary production uses the author model. No inspected call requests weight training. These are local wiring findings, not a universal absence of provider-side changes (src/agent/models.ts, src/agent/server.ts, src/lib/env.ts, src/ingest/embedder.ts, src/search/rerank.ts).
Retained material and later consumers
MEM-OBJ-chunks, MEM-OBJ-page-revisions, MEM-OBJ-revision-sources and MEM-OBJ-wiki-chunks preserve the distinctions between source text, derived numerical/symbolic access structures, revision prose/history and citation tuples. Schema and store reads support their storage and lineage. Repo deletion removes citation rows without rewriting inline prose. The index is replaceable, not an archive of every Git snapshot (src/db/schema.ts, src/wiki/store.ts, src/ingest/repos.ts, src/ingest/sync.ts).
MEM-RTE-index-page is genuinely a segment of revision admission, so its Part of: relation is supported. I followed embedding and transactional replacement rather than stopping at the currentness guard: the guard precedes the remote call and is not repeated inside replacement. MEM-RTE-search-knowledge correctly describes scoped rank fusion and hydration, including matching embedding IDs only on vector candidates and repo filtering through indexed-revision citations. Hydrated wiki hits omit revision/stale fields. Thus the reports preserve index lag/race and delivery limits without pretending that accepted persistence guarantees current search text (src/ingest/index-page.ts, src/search/index.ts).
MEM-OBJ-chat-messages, MEM-OBJ-chat-summary, RT-RTE-context-fold and MEM-BAP-summary-context support a real trace-fed write and later instruction consumer. I checked boundary selection, clipped tool transcript, prior-summary input, persistent cutoff and chat instruction composition. Old rows remain stored; the model receives summary plus selected tail. Regeneration truncates messages without resetting the summary/cutoff, and a missing cutoff reactivates supplied history. The reports correctly withhold semantic preservation, hard total-context guarantees and demonstrated learning (src/agent/context.ts, src/agent/chat-store.ts, src/agent/prompts.ts, src/agent/chat.ts).
RT-OBJ-wiki-schema, RT-BAP-schema-instructions, MEM-OBJ-project-context and MEM-BAP-project-brief are distinct instruction paths: per-call filesystem policy versus saved owner intent/layout. Formatter conditions and validated saves support the known replacement/read routes without establishing installed-state authorship or model compliance (WIKI_SCHEMA.md, src/agent/prompts.ts, src/project/context.ts, src/project/actions.ts).
RT-RTE-external-consumption matches MCP registration/formatting and export. I followed the actual delivered text: compact-profile deduplication and clipping, proposal-only read fallback, link rewriting and freshness labels differ from accepted reachable-tree export. The application page also previews proposal-only bodies, while store/chat page reads have empty unpublished bodies. These are compatible findings, not contradictory review guarantees (src/mcp/server.ts, src/mcp/handlers.ts, src/export/export.ts, src/app/w/[wiki]/wiki/[...path]/page.tsx).
MEM-OBJ-wiki-log and MEM-OBJ-eval-results distinguish diagnostics/accounting from synthesized production memory. Stored reasons are not automatically a criticism prompt. Citation evaluation's resume reader is real but does not install a production policy (src/db/schema.ts, src/agent/runs.ts, src/agent/author.ts, scripts/eval-citations.ts, scripts/bench-mcp.ts).
Epistemic routes, evaluation and forcing alternatives
The epistemic objects—EPI-OBJ-source-evidence, EPI-OBJ-page-prose, EPI-OBJ-source-citations, EPI-OBJ-chat-content and EPI-OBJ-evaluation-output—separate truth-apt content from coordinates, opaque generation and diagnostic containers. The grounder's replacement branch and source parser support the illustrative link-removal deduction without executing it. The ledger does not mistake that deduction for a witnessed model output. Its indeterminate generation/summary classifications and operational-acceptance versus factual-acceptance distinction are warranted.
I checked EPI-CLM-wiki-grounding against README/schema wording and all material admission alternatives above. The narrower implemented conclusions preserve both the useful reference filter and its limits. EPI-CLM-lint-contradictions is a real schema request; the complete deterministic linter implements the five reported structural classes, not semantic contradiction evaluation. MEM-RTE-lint-wiki and EPI-RTE-wiki-lint describe the same scheduled/live mechanism. Logs versus live recomputation and advisory rather than automatic revision force are supported (README.md, WIKI_SCHEMA.md, src/wiki/lint.ts, src/wiki/jobs.ts).
RT-RTE-explain-answer correctly separates selected-key rehydration and reference filtering from ordinary chat. RT-RTE-retrieval-eval computes fixed expected-path metrics, can change source state and does not admit a winning production strategy. RT-CLM-provider-routing survives the alternate-path challenge: the UI explainer locally rejects reranking with ZDR enabled, while the evaluation directly invokes reranking with provider preferences. Remote rejection or retention is not established. Disabled ZDR and injected dependencies also prevent a universal all-calls privacy guarantee (src/agent/explain-answer.ts, src/search/explain.ts, src/search/rerank.ts, eval/run.ts, src/lib/openrouter-privacy.ts).
MEM-RTE-bench-author and EPI-RTE-benchmark-checks correctly retain whole-wiki destruction, caught author failure followed by exit zero, apply-mode chat tools with a human label, and MCP's path-substring recall proxy. These shipped callers were not treated as external just because they invoke production APIs. The MCP driver's restrictions are host arguments, not proof of executed isolation (scripts/bench-author.ts, scripts/bench-chat.ts, scripts/bench-mcp.ts).
I read the complete citation evaluator to verify EPI-RTE-citation-evaluation: extracted cited sentences, synthetic nonoverlap decoys, commit-specific excerpts, model support criteria, quote flags, resume reuse and output consumers. This is an implemented semantic assessment request outside production admission; quote validity does not veto the verdict, and no verdict installs a revision. EPI-CLM-evaluation-results correctly treats published numbers as reported operation and conditions 87% on the agreed subset. Independent truth labels, executed comparisons and component causal effects remain unestablished (scripts/eval-citations.ts, README.md, docs/evals.md).
Reconciliation and omission-search stopping decision
The reconciliation correctly supersedes the duplicate epistemic lint route with the memory lint route without replacing either finding. Its other overlap dispositions preserve valid containers, projections and distinct substrates; they do not require a split supersession. The index segment is the sole explicit containment relation. Epistemic content projections and memory's containing state are not interchangeable whole records. Workflow annotations extend existing routes rather than redeclaring them. No evidenced absence record needs a search-boundary repair. No unresolved cross-report contradiction was found: differing evaluator inspection depth, proposal visibility, index synchronization and summary rollback are explicitly compatible qualifications.
Consequential omission questions were selected from the frozen tracked tree, manifest, shipped callers and producer/consumer dependencies: whether another write caller bypasses review/grounding; whether a consumer silently substitutes proposed or stale text; whether folding has a separate reload/rollback route; whether offline evaluators perform semantic checks or install production changes; and whether a privacy guard covers every rerank caller. Source searches across src/, scripts/ and eval/, followed by the complete implementations discussed above, give each question a supported disposition. Additional checks of src/app/api/w/[wiki]/chat/page/route.ts and src/mcp/instructions.ts show an approval-card body reader and advisory external-host selection guidance, not another production admission or history-reload mechanism.
Further optional expansion stopped because no identified material challenge lacks a disposition and the emitted findings make no exhaustive security, evaluator-quality or deployed-operation claim. Remaining uninspected support code is not evidenced absence. Existing reports already express provider/host opacity, unavailable installed state, unobserved activation/benefit and selective coverage beside the affected findings. Those are faithfully scoped uncertainties, not additional defects to convert into blockers or duplicate as new limits. The supported architectural account does not imply complete route enumeration, reliable model compliance, semantic citation accuracy or exercised self-improvement.
Blockers
none
Limits
none