Secure LLM-Wiki report verification

Type: agentic-system-analyses/types/agentic-system-verification.md

Verification

The runtime, memory, epistemic, workflow and reconciliation members support a bounded implementation account. I independently read the source supporting their consequential findings rather than treating their reported checks as verification. The supplied structural report contains none; all four correction-answer inputs are empty. There are no structural failures to route, correction claims to accept, or declined objections to adjudicate.

Boundary, records and evidence

The frozen checkout HEAD equals the reviewed revision and its working tree is clean. I enumerated the tracked tree and compared it with the boundary coverage. Packaging leads to the shipped driver, not an excluded host. Direct inspection covered the complete driver, extractor, sanitizer, gate, reviewer, models, prompt builders, both stores, context loader, trust registry and provider adapters. Additional reads covered the README, roadmap, packaging, corpus evaluator, selected CLI and read-hygiene test definitions, and the article's security/future-work discussion. These are SRC-1 implementation or doctrine/design at the registered commit; none is observed-run or causal evidence. No target command, test or provider was executed.

I checked every declared record's referent, source anchor, evidence interpretation and status, and each annotation's consumer-specific claims. The three model components distinguish external parameter storage from emitted text/vectors, mutable configuration from pinned artifacts, and application calls from opaque provider behavior: RT-CMP-extraction-model, RT-CMP-review-model, RT-CMP-embed-model. SRC-1, src/secure_wiki/llm_client.py, supports those distinctions. The static RT-OBJ-system-prompts and RT-BAP-prompt-policy correctly describe requested model behavior rather than enforced obedience. Their quoted policy passages support the stated scope, not effectiveness.

Invocation, alternate paths and effects

I traced RT-RTE-extraction-parsing and RT-RTE-ingest-admission through their producers and consumers in SRC-1, src/secure_wiki/__main__.py, src/secure_wiki/extraction/extractor.py, src/secure_wiki/gate/write_gate.py and src/secure_wiki/review/adversarial.py. Original transformed/truncated source text reaches extraction and review despite sanitizer flags. Whole-envelope nonce checking does not cover accepted bare arrays or linewise extraction. Free-text provider completions can reach those alternatives without a special caller; they still undergo downstream review and gates. The reports correctly withhold both universal extraction verification and a demonstrated unreviewed-persistence exploit.

The batch reviewer sees source and marker-stripped proposed/existing trusted text, not proposed trust metadata. It is invoked before the per-claim loop and reused while successful claims/vectors update subsequent gate inputs. Fail-fast gate ordering, global untrusted-versus-existing-trusted rejection, similarity thresholds and section fallback match the accounts. Direct gate calls can supply review results or invoke review without source, whereas the shipped CLI supplies its batch review. Review nonce/JSON retries are bounded, not general exception recovery. Non-object JSON fails the envelope predicate; unvalidated result-field values and provider exceptions still prevent treating this as comprehensive malformed-response handling. This precision does not undermine the reports' bounded recovery conclusion.

I traced both context implementations, not only the strict builder, for RT-RTE-context-query and RT-BAP-query-evidence. SRC-1, src/secure_wiki/read/hygiene.py, src/secure_wiki/prompts.py and src/secure_wiki/__main__.py, support the hard untrusted-label exclusion in both paths, pending inclusion only in the actual loader when requested, and delivery inside the query system string. The pending-page test definitions demonstrate supported stored input construction, not successful ordinary-ingest bypass or an observed run. Selection precedes the question; no vector retrieval, accumulated dialogue, answer checker, answer persistence or session refresh is wired into this route.

For RT-RTE-operator-maintenance, direct storage and reset branches were checked against SRC-1, src/secure_wiki/store/wiki_store.py and src/secure_wiki/trust/tiering.py. Save does not enforce gate passage or active status. Files precede Git commits; vector writes are separate. Broad staging defeats a universal untracked-vector assertion. Default reset destroys history; preserved history is an alternative, not append-only enforcement. These counterpaths are already retained rather than hidden behind normal-path guarantees.

Memory and epistemic findings

The source supports the memory parts and their separate consumers: MEM-OBJ-claim-text, MEM-OBJ-claim-metadata, MEM-OBJ-gate-records, MEM-OBJ-claim-embeddings, MEM-OBJ-trust-rules and MEM-OBJ-wiki-history. Frontmatter supplies reload text; body-only detail does not. Gate names reach context but failed review reasons do not. Vectors affect later admission independently of a claim-status join and carry no model-version marker; they are not query retrieval or parameter learning. Rule edits affect future assignment, not automatic retrospective re-tiering. Git inspection/reversal is an operator affordance, not an automatic model read-back route. The source passages retained in these records establish those mechanisms without establishing activation or benefit.

The epistemic objects preserve the same distinctions: EPI-OBJ-source-text, EPI-OBJ-claim-text, EPI-OBJ-claim-metadata, EPI-OBJ-review-result, EPI-OBJ-claim-embeddings and EPI-OBJ-query-answer. No supplied completion or retained instance warrants stronger semantic classifications. EPI-RTE-security-checks correctly bounds the evaluator domains; cosine similarity and section matching do not establish contradiction. EPI-RTE-gate-disposition is operational admission, not factual acceptance. EPI-RTE-store-retention establishes retention without inventing subsequent factual integration. EPI-RTE-text-render supplies a valid source-only counterexample to universal semantic preservation: the regex removes the very token mentioned by its example sentence. This is neither an executed observation nor proof that every rendering changes meaning.

The doctrine comparisons are supported for RT-CLM-security-boundary, EPI-CLM-compiled-knowledge and EPI-CLM-quality-evidence. Their sources declare trust protection, compilation and evidence quality; the implementation establishes narrower mechanisms. The reports do not promote those claims into security certification, factual warrant, demonstrated learning or self-improvement.

Workflow, relations and omission disposition

The workflow account follows the real persisted-state handoff, with a separately triggered query rather than an invented automatic continuation. Model tasks return text; Python owns sequencing, veto interpretation and writes. Escalation terminates in quarantine without an approval consumer. Streamed stdout is delivery, not independently accepted correctness. Recovery qualifications agree with the inspected exception and persistence boundaries.

I checked every Part of: relation against the declared endpoints and operative parts. Input source, output text/metadata/vectors, review evidence and check/disposition subroutes fit the ingestion container; query answer fits the query route; retained configuration fits maintenance. Cross-stage history, rendering and storage appropriately avoid claiming exclusive containment. The reconciliation correctly supersedes only the two same-referent text/vector duplicates, retaining both declarations and their qualifications. Its broader epistemic metadata container remains compatible with memory's finer diagnostic split. Claim, carrier and authority-path records are not duplicates. Every memory integration issue has a disposition; no unresolved conflicting finding or missing required part was identified.

Consequential omission questions were selected from the tracked tree and state dependencies: could a shipped caller evade the reported context policy; could reload reinterpret a different retained part; could another writer/evaluator complete promotion, supersession or feedback; could vector/history maintenance justify stronger consistency or recovery claims? Searches across SRC-1 src/ and selected tests/docs, followed by complete producer/consumer reads, supported the retained alternate-path account. MEM-ABS-query-feedback is warranted within the shipped query path: the complete loop streams and prints, with no store update, and the roadmap marks feedback as future work. Schema successor fields likewise lack an automatic writer in the inspected driver/gate. The corpus evaluator supplies fixed claims and manifest-derived mocked reviews, not live extraction/reviewer efficacy. Other CLI tests also mock model stages; their presence does not supply missing run evidence.

Further omission expansion stopped because all selected challenges have supported dispositions and no identified material alternative remains unresolved. Additional payload-by-payload efficacy analysis or external-provider investigation would not justify broader conclusions from this source-only boundary. Actual provenance, faithful extraction, model compliance, service behavior, operational vector compatibility, factual accuracy and benefit remain expressly unestablished in the records. That faithfully scoped uncertainty is not a new verification defect or an additional publication limit.

Blockers

none

Limits

none