Secure LLM-Wiki synthesis

Type: agentic-system-analyses/types/agentic-system-synthesis.md

Bounded synthesis

Source compilation under host control

Secure LLM-Wiki implements a persistent source-to-claim-to-answer pipeline. Python owns progression and filesystem effects; models propose statements, judge manipulation and generate answers through text calls. It is not a model-selected tool loop. The declared product is a curated knowledge wiki rather than repeated retrieval of raw documents. The implementation supports persistent candidate compilation and reuse, not demonstrated factual knowledge production (RT-RTE-ingest-admission, EPI-CLM-compiled-knowledge).

The evidence is SRC-1, the repository https://github.com/nicobleh/secure-llm-wiki at commit dfbc60e37487b61e9a5cbce1271a13576a776d51, with an applicability cutoff of 2026-10-19. The whole-system boundary includes the shipped CLI, prompts, admission, persistence, maintenance and query consumers. Code and documentation were inspected; no target commands, tests or models were executed. External provider internals and generated instance state are not supplied. Thus the account establishes implementation and declared intent, not successful operation or causal benefit (boundary and source register). Both independent verifications report no blockers or additional publication limits (report verification, profile verification). This synthesis adds no source reads or evidence.

The representative workflow is an operator-selected ingestion followed by a separately invoked query. The connection is persisted pages, not an automatic continuation. The installed driver exposes init, ingest, list, context, query and clear; the local process user supplies its operating authority. Models receive no filesystem tools through these calls (workflow account, RT-RTE-operator-maintenance).

Acquisition, proposal and admission

An operator chooses a file, URL or folder and may override source trust. Python loads existing claims and embeddings, transforms acquired text and keeps its first 8,000 characters. The source hash covers the normalized processed excerpt, not the complete original. Sanitization produces flags and cleaned text, but extraction and review receive the original truncated text. Flags can later reject writes; they do not prevent initial model exposure. Extraction receives a fresh system instruction and nonce-delimited source, without the existing wiki or extraction history. Its parser constructs pending claims with caller-supplied provenance and trust (RT-RTE-extraction-parsing, EPI-OBJ-source-text).

Envelope-shaped whole JSON must echo the nonce, but accepted legacy arrays and linewise JSON do not uniformly undergo that check. These alternatives are reachable from ordinary free-text model responses. They still proceed through review and admission, so the gap does not demonstrate unreviewed persistence. Empty extraction ends without a quarantined claim (RT-RTE-extraction-parsing).

The host next requests embeddings and one review for the source's entire proposal batch. Embedding failures can leave partial vector coverage or select a non-vector fallback. Review receives the source, proposed text and existing active trusted text in a separate context. Separate conversations are wired; different model weights are not enforced. The reviewer is explicitly asked to detect manipulation, not determine truth. Its batch result is reused while Python processes claims sequentially and updates existing state after successful writes (RT-RTE-ingest-admission, RT-CMP-review-model, workflow account).

Five ordered checks govern ordinary CLI admission: sanitizer flags, provenance completeness, trust tier, review verdict and consistency. The first failing check determines disposition. Any existing active trusted claim blocks an untrusted proposal at the trust check, without establishing a shared subject or overwrite. Otherwise an untrusted claim can become active, although it remains ineligible for shipped query context. Consistency uses maximum cosine similarity with default duplicate/escalation thresholds of .95/.85; without usable vectors it uses same-section/different-source trusted claims. Neither operation proves contradiction. The vector comparison does not reconcile its entries with current claim status, trust or model version (EPI-RTE-security-checks, MEM-OBJ-claim-embeddings).

Passing all checks permits active page storage without per-proposition human approval. Quarantine and escalation both terminate in quarantine; “human review required” does not create an approval inbox or automatic promotion route. A valid blocking review returns immediately. Invalid review JSON or nonce mismatch triggers at most three calls with the same prompt and nonce, but this is not general provider-error recovery (EPI-RTE-gate-disposition, workflow account).

Durable handoff and later reliance

Claim text and metadata persist in YAML-frontmatter Markdown; reload reads frontmatter, not the duplicated prose body. A body-only edit therefore does not revise the consumed claim. Direct storage calls and manual frontmatter edits can supply records without the ingestion gate. This is a local/API control boundary, not evidence of a remote unauthenticated endpoint. Files are written before Git commits, and embeddings are saved separately afterward, so partial failure can leave mixed state (MEM-OBJ-claim-text, EPI-RTE-store-retention).

Retained parts have different later effects. Active trusted text supplies reference context for later review; labels govern code eligibility and fallback checks; vectors rank similarity matches and enforce threshold outcomes. Embeddings are admission memory, not a semantic query index. Failed-review reasons and selected diagnostics survive storage, but ordinary review/query do not receive those reasons. Git history affords human inspection and reversal where history survives; no automatic model consumer learns a new policy from it (MEM-OBJ-claim-metadata, MEM-OBJ-gate-records, MEM-OBJ-claim-embeddings, MEM-OBJ-wiki-history).

A later query or context requests retained pages. Code selects every eligible claim by stored trust/status, before receiving a question, without question-specific retrieval or explicit context-budget trimming. Default selection is active trusted/semi-trusted pages; --include-pending can also deliver pending pages. Strict alternate builders exclude pending. Both paths exclude claims whose loaded tier is untrusted, even when the requested minimum is untrusted; quarantine is separately excluded by directory. These controls trust current deserialized metadata rather than authenticating its history. Pending delivery requires suitable stored pages and is not a demonstrated ordinary-ingestion bypass (RT-RTE-context-query, MEM-OBJ-claim-metadata).

The loader strips literal trust-marker tokens from claim bodies and renders text plus source, section, time and gate names inside nonce-delimited system context. Token removal is not universally meaning-preserving, and copied provenance is not authenticated. The query instruction requests evidence-only answers, citations and abstention, but no independent comparator checks the answer. Each question receives the same startup snapshot plus only the current question, not earlier turns. Answers stream to stdout and are not saved. This is delivered text, not independently accepted correctness (EPI-RTE-text-render, RT-BAP-query-evidence, MEM-ABS-query-feedback).

Maintenance changes future loads, not an open query snapshot. Operator rule edits do not retrospectively re-tier claims. Clear/reset can remove working state; default reset destroys Git history while preserving trust rules, whereas history-preserving deletion is another route. Recovery is manual inspection, retry or Git reversal where possible, not atomic rollback or automatic poison repair (MEM-OBJ-trust-rules, RT-RTE-operator-maintenance).

What these mechanisms establish

For the scenario of reusing operator-selected documents across sessions, the strongest supported contribution is a wired durable handoff: model-proposed statements become retained inputs to later review, admission and answering. For security-sensitive reuse, the implementation supplies several distinct barriers, not a universal guarantee that source instructions cannot influence later answers. For fact-dependent answering, security admission is not factual acceptance: populated provenance, assigned trust, model manipulation review and similarity checks supply no proposition-level truth criterion. Without actual completions, extraction and answer generation remain semantically indeterminate rather than proven faithful reshaping, derivation or conjecture (RT-CLM-security-boundary, EPI-RTE-gate-disposition, EPI-CLM-quality-evidence).

Persistence can expand what later calls receive; improved capacity for future action has not been demonstrated. No comparison measures the benefit of compilation, any component's causal contribution or criticism-driven improvement. The searched query route specifically lacks a feedback-to-store update. That local negative does not establish a system-wide absence of trace learning: original document activity-trace provenance, arbitrary direct writes and opaque provider updates remain unresolved (MEM-ABS-query-feedback, MEM-OBJ-claim-text, profile verification).

Theory-builder conditions also remain separate: (1) no actual formulated explanatory theory instance is supplied; (2) content-dependent consumption of such a theory is not established by candidate delivery; (3) manipulation review and similarity checks do not establish content-directed theory criticism with blame and changed reliance; (4) sequential reuse does not establish retention and iteration of such criticism. The records therefore do not establish a theory-building episode or improved future capacity from criticizing a consumed theory. Reflection is not established by retained diagnostics, which have no shown representation-mediated automatic policy-update loop. Autonomous theory building is not established, although computation owns the internal stages of ordinary ingestion after operator initiation. Self-improvement is likewise unestablished over the ingestion-to-later-query horizon: retained output has a live delivery path, but evidence-responsive determination of an improvement-directed organizational update and subsequent causal exercise of that update are not shown. These are independent conclusions, not a single negative grade (RT-RTE-ingest-admission, RT-RTE-operator-maintenance, MEM-OBJ-gate-records, RT-RTE-context-query).

The assessment would change with source-linked live extraction/review/query traces, factual-support and attack evaluations using actual providers, and controlled comparisons addressing benefit. Uniform nonce enforcement, admission at storage boundaries, checked vector lineage, an evidence-consuming factual acceptance route or an implemented evaluation-fed update would change the corresponding architectural findings. None alone would establish successful operation or causal improvement.

Limitations

Limitation Affected SRC IDs and record citations Inspected boundary Conclusion prevented Evidence that would resolve it
Source-only evidence; mocked corpus review is not live efficacy measurement. SRC-1; RT-RTE-ingest-admission, RT-BAP-query-evidence Shipped driver, prompts and evaluator definitions; no executed target sessions. Successful operation, model obedience, injection resistance, factual accuracy or causal benefit. Registered end-to-end traces and live evaluations with explicit conditions and suitable comparisons.
No generated instance or authenticated deployed provenance/configuration. SRC-1; MEM-OBJ-claim-text, MEM-OBJ-claim-metadata, MEM-OBJ-trust-rules CLI producers, direct storage, frontmatter reload and policy loading. Actual retained content, effective trust, source support, or human approval of arbitrary API writes. Frozen instance/configuration and attributable production, edit and approval records.
No supplied completions or proposition-level factual acceptance. SRC-1; RT-RTE-extraction-parsing, EPI-RTE-gate-disposition, EPI-RTE-text-render, EPI-OBJ-query-answer Acquisition, parsing, rendering, admission and unchecked answer output. Faithful extraction, warranted factual reliance, or classification of actual output as entailed or ampliative. Source-aligned candidate/output instances and evidence-consuming checks against named factual criteria and scope.
Vector compatibility, completeness and operational effects unresolved. SRC-1; MEM-OBJ-claim-embeddings, EPI-RTE-security-checks JSON reload, cosine comparison and fallback; no model/version or claim-state reconciliation. Complete semantic consistency coverage or reproducibility after model changes. Provenance/version-linked vectors, reconciled claim state and observed comparison outcomes.
External providers and unspecified API consumers remain opaque. SRC-1; RT-CMP-extraction-model, RT-CMP-review-model, RT-CMP-embed-model, RT-RTE-context-query Application adapters and alternate builders, not external internals or deployment. Enforced independent weights, pinned artifacts, provider retention/update classifications or complete external read-back inventory. Exact deployed artifacts/contracts and inspectable provider or external-consumer evidence.
Partial persistence and recovery are not exercised. SRC-1; EPI-RTE-store-retention, MEM-OBJ-wiki-history Separate file/Git/vector effects and destructive/history-preserving maintenance branches. Atomic delivery, append-only audit or reliable recovery. Failure/recovery traces and, for stronger guarantees, transactional or protected-history enforcement.
Learning and self-improvement chains are not established by retained candidates or diagnostics. SRC-1; MEM-ABS-query-feedback, MEM-OBJ-gate-records, RT-RTE-ingest-admission, RT-RTE-context-query Shipped source compilation, stdout-only query and diagnostic reuse; original trace provenance and provider adaptation unresolved. System-wide trace-learning negative, demonstrated improved capacity, theory-criticism-driven improvement or exercised self-improvement. Qualifying original inputs, retained update and later consumer traces, objective-bearing update evidence and causal comparisons at the claimed boundary.