LLM-Wiki-v3 synthesis

Type: agentic-system-analyses/types/agentic-system-synthesis.md

Bounded synthesis

From source documents to proposed pages

LLM-Wiki-v3 is a Markdown knowledge store with model-assisted extraction, curated admission, retrieval and evaluation. It is not an enclosing agent runtime. Its declared audience includes human browsers and agents, but the shipped query path returns retrieval metadata, not an assembled answering prompt or an agent answer. This account covers the whole shipped repository at commit 267541db838cbca757a112bd0cfa47fa61eb5fb2, with an applicability cutoff of 2026-10-18. Its sole source, SRC-1, supplies inspected implementation, design guidance and attributed evaluation reports. No target code, tests, provider calls or end-to-end workflow were executed; the saved baseline is reported operation, not an independently observed run or causal experiment (boundary, Source register).

The representative progression is an operator's document ingestion, software staging, human review, explicit index rebuild and later query. These are separately invoked commands connected by files, not an automatically completed task. Python owns progression. The model proposes page content; software checks and selects proposals; the interactive reviewer decides admission; the operator initiates rebuild and retrieval. Completion is distinct at each stage: a staging manifest, a retained destination, a build manifest and ranked retrieval metadata (workflow account).

Ingestion retains original source bytes, divides decoded text into overlapping chunks and supplies each chunk to an Anthropic extraction call with root ingest instructions and a structured proposal schema. Each call starts fresh. Existing corpus bodies, earlier proposals and rejected-attempt critiques are not added to its context. The host retains full source text for quote location, but the model has no source-reload tool. This controls context by isolating chunks and returning proposals rather than accumulating a conversation; the approximate chunk target is not a strict token-window limit (RT-RTE-source-ingest, memory context annotation). The structured proposal request does not grant the model the host process's filesystem, editor or Git capabilities.

Code locates proposed quotes, constructs source pointers and checks schema conformity and overlap between marked claim fragments and source spans. The first nonempty all-valid set wins per chunk. Otherwise a stronger extractor receives the same selected input, not a diagnosis of the first attempt; the final attempt supplies the pages considered for staging. Valid pages from an earlier mixed result are not merged into the retry. First-ID deduplication and issue-free staging then determine pending files (EPI-RTE-proposal-selection, workflow retry annotation).

These checks enforce bounded eligibility, not truth. Quote location establishes textual correspondence; token overlap does not establish entailment or source credibility. Citation loops enumerate existing markers rather than every substantive assertion, and do not check all prose metadata. The ingest metric even returns its maximum when it parses no marked claims, although the CLI separately skips pages lacking sources or loadable raw text. Thus neither a passing proposal nor a high metric warrants all-assertion semantic support. Whether generated pages preserve meaning or introduce unsupported propositions remains indeterminate without outputs and semantic comparisons (RT-CLM-provenance-policy, EPI-RTE-span-check, EPI-RTE-ingest-measurement, EPI-RTE-page-extraction).

Admission creates reusable state, not a universal approval guarantee

Interactive review displays an 800-character preview, optionally opens an editor and asks for accept, reject, skip or quit. Acceptance moves the current file to its registered namespace and optionally commits it. It does not rerun schema or span checks after editing. The optional installed Git hook checks structure after movement, not semantic support, and the supported no-commit route avoids that commit check. Callable acceptance does not require proof of a human decision. The shipped seed driver is another producer: it validates authored specifications and writes directly, without per-page interactive approval (RT-RTE-page-review, RT-RTE-curated-seed, EPI-RTE-schema-check).

This makes admission route-specific. Operational approval is not established evidence-consuming acceptance of a truth claim against a named criterion, intended use and scope. Retaining pages, adding links and building indexes therefore do not establish lifecycle integration of accepted knowledge (EPI-RTE-review-disposition, EPI-RTE-knowledge-retention).

The store retains prose, symbolic routing fields and source-span pointers, but not the model's selection rationale. Ingest diagnostics persist separately; review enumerates Markdown and does not automatically present manifest warnings. Same-name acceptance or seed writing can replace retained content, despite instructions to keep old pages through supersession. Rejection deletes the pending page, not its raw source. Raw writes, page moves and independently replaced index databases are not one transaction; optional Git history and manual rebuild are recovery resources, not automatic rollback (MEM-OBJ-page-body, MEM-OBJ-page-sources, MEM-OBJ-ingest-manifest, RT-CLM-history-policy, runtime recovery account).

Later consumption determines what the retained state can do

Explicit rebuild loads nonsuperseded pages into lexical, graph and optional dense SQLite derivatives. Query combines component rankings through reciprocal-rank fusion and optionally reranks with a cross-encoder. This is relevance selection, not a generative answer or truth endorsement. Retained prose and aliases shape ranking; relation fields shape graph traversal; dense vectors supply numerical similarity inputs without training model weights (RT-RTE-index-query, MEM-OBJ-bm25, MEM-OBJ-graph, MEM-OBJ-vectors).

Admission does not rebuild indexes. Default current-page filtering during loading and rebuild is real, but ordinary fused output does not universally exclude stale or missing IDs. Active reranking filters candidates against current pages; unavailable reranking can fall back to the unfiltered branch. Query output contains IDs, scores, contributing paths and source IDs, omitting bodies and stored spans. Currentness and intact provenance delivery therefore cannot be inferred from the output alone (RT-RTE-index-query, EPI-RTE-query-selection).

There is a concrete later memory-to-model route, but it generates evaluation questions rather than answers. On a bootstrap request, code automatically enumerates registered pages, including superseded ones, and supplies each retained body to a fresh model invocation. This input delivery is coarse push selection: the model has not requested that body. It is distinct from the upstream operator request and from request-led query retrieval. No explicit input-body truncation budget is supplied; the response cap does not bound retained input. Frontmatter, raw evidence and provenance arrays are omitted (RT-RTE-query-bootstrap, MEM-OBJ-page-body, profile verification). Code assigns the current page ID to each generated question without semantic label validation. Default output requests later human review and merge, but a caller-selected active-dataset path can write directly where the evaluator reads.

Evaluation has separate consumers. The baseline retriever loads current corpus text without persisted indexes; component evaluation inherits index-query limits; ingest evaluation depends on available raw evidence. Expected page IDs define retrieval scoring, while source overlap defines the ingest statistic. These numerical derivations support dataset-label agreement and lexical-pass claims, not answer truth, transfer or isolated component benefit. The implemented comparison reports rather than automatically promoting changes (memory evaluation annotation, EPI-RTE-retrieval-measurement, EPI-CLM-eval-gates).

What this establishes about future capacity

The strongest supported contribution is reusable source-attributed content, multiple implemented later consumers and operator-facing measurements. Validation-triggered escalation can alter the current extraction choice. These mechanisms provide more than passive storage, but no supplied comparison establishes that criticism of a consumed theory improved capacity for future action. Retry does not carry a formulated criticism forward, and retained audit or evaluation results have no inspected automatic policy-update consumer (EPI-RTE-proposal-selection, RT-RTE-eval-report, MEM-OBJ-audit-log).

The supplied runtime does not establish a formulated-theory route meeting theory-builder conditions 1–4: localized instructions and page claims alone do not establish a theory unit; instruction delivery alone does not establish decisions depending on its theoretical content; lexical scores and model switching do not establish content-directed criticism; retained diagnostics alone do not establish a next round shaped by criticism. These are separate unestablished conclusions, not evidenced absences about opaque model internals. Reflection is independently unestablished: the records do not show both causal directions between a self-representation and the represented system. End-to-end autonomous theory building is also unestablished, separately from the automatic substeps and the interactive human admission route (RT-BAP-ingest-instructions, epistemic bounded conclusion).

At this shipped-system boundary, with later ingestion or retrieval as the horizon and improved extraction/retrieval as the objective, exercised self-improvement is not established. Metrics do not automatically determine organizational updates, and no subsequent operation is shown to depend causally on such an update. Page or dataset revision can affect later consumers, but its label and persistence do not close those missing links or demonstrate improvement. Trace-fed learning is likewise unresolved rather than globally absent: document/page inputs do not establish qualifying trace origins, and raw process-record retention does not establish a durable behavior-shaping update (RT-RTE-query-bootstrap, MEM-OBJ-eval-results, profile verification).

For operator-managed source extraction and browsing, the implementation establishes the staged workflow and reusable retrieval state. For semantic acceptance, reliably current withdrawal or agent-answer benefit, the evidence stops earlier. End-to-end traces connecting source bytes, proposals, reviewer evidence use, rebuild state and later consumption would change the exercised-use assessment. Semantic comparisons would change the fidelity assessment. Universal post-edit revalidation or query-wide current-ID filtering would change the corresponding architectural guarantees. Evidence-responsive retained updates plus later causally dependent operation would be needed for an exercised self-improvement finding; controlled outcome comparisons would separately be needed for successful improvement. Independent report and profile verification found no remaining blockers or declared limits; that acceptance does not upgrade implementation evidence to operation or benefit (report verification, profile verification).

Limitations

Limitation	Affected SRC IDs and record citations	Inspected boundary	Conclusion prevented	Evidence that would resolve it
No observed end-to-end operation or causal comparison	SRC-1; RT-RTE-source-ingest, RT-RTE-page-review, RT-RTE-eval-report	Shipped implementation and source-authored saved reports; no target execution	Successful review/delivery, reproduced baseline results and causal benefit	Linked execution artifacts and comparisons with stated interventions, outcomes and confounding limits
Missing retained raw/index artifacts and incomplete corpus-content audit	SRC-1; MEM-OBJ-raw-source, MEM-OBJ-page-body, MEM-OBJ-vectors	Writers/loaders inspected; raw bodies and derived instances unavailable; most corpus bodies and individual seed specs not audited	Corpus-wide provenance replay, content quality or actual numerical-state conclusions	Retained bytes/index artifacts with lineage and a scoped content/seed audit
Generation and human judgment are not semantically established	SRC-1; EPI-RTE-page-extraction, EPI-RTE-review-disposition, EPI-RTE-query-bootstrap	Prompt/check/review/bootstrap implementation, without actual responses or evidence-consuming review records	Faithful extraction, accepted ampliative knowledge, semantic label validity or justified truth acceptance	Generated candidates compared with sources and recorded review decisions naming evidence, criterion, use and scope
Initial dataset control and external model identity remain unresolved	SRC-1; MEM-OBJ-eval-queries, RT-CMP-haiku-extractor, RT-CMP-sonnet-extractor, RT-CMP-dense-embedding, RT-CMP-cross-encoder	Known subsequent dataset transformations and configured model call sites; external weights/services excluded	Complete lineage/admission classification, exact model fixity, provider reliability or internal learning conclusions	Dataset provenance/admission history and separately authorized model artifacts/service evidence
Intended answering-agent integration is not supplied	SRC-1; RT-RTE-index-query, RT-RTE-query-bootstrap	Query returns metadata; bootstrap consumes bodies for evaluation-data generation; downstream answering excluded	Deployed MCP/context compilation, evidence delivery to an answering agent or improved answers	Consumer implementation and traces of selected content reaching and affecting later answers
Future-capacity and self-change links remain open	SRC-1; EPI-RTE-proposal-selection, MEM-OBJ-audit-log, MEM-OBJ-eval-results, RT-RTE-eval-report	Retry, diagnostics, reporting and inspected later consumers; no bounded absence records	Improved future capacity, complete trace-learning classification, theory criticism/iteration, reflection or exercised self-improvement	Property-specific retained-update and consumer evidence; for exercised self-improvement, objective-responsive determination and later causally dependent operation; separate outcome evidence for success