llmwiki-marimo synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From documents to reusable pages
LLM Wiki is a local knowledge system for ingesting documents, browsing retained pages and answering questions from them. This account covers the shipped web and marimo applications, generation and chat services, prompts, maintenance and evaluation drivers at Git commit 0df06b0499a6242a4a5edfb1495e1d55a3e3c8b7, with an analysis cutoff of 2026-10-18. SRC-1 supplies implementation and source-native design evidence, not an observed workload or causal experiment. External provider internals, weights and deployed user workspaces are outside the inspected evidence. The whole-system functional boundary is not an exhaustive algorithm audit (boundary, Source register).
The distinguishing progression is source acquisition → generated wiki → later retrieval and merging. Upload or scan starts a code-owned pipeline. It extracts and commits source text before requesting model-generated concepts and summaries. Existing same-slug concept bodies become input to later generation. Software then admits generated Markdown and updates references, search state and collection pages without per-generated-page human approval. A malformed extraction response can become a raw summary fallback rather than being rejected. Instructions to preserve existing content guide the model; they do not enforce a lossless merge (RT-RTE-document-ingest).
Editable Markdown is the durable knowledge product. SQLite retains source text, lexical search chunks, references and diagnostics; it is not a vector-memory design in the inspected paths. Page prose supplies interpreted context, while IDs, FTS terms, aliases and thresholds have fixed program consumers. Original sources remain a separate evidential substrate. Citation connections establish lineage, not source truth or semantic preservation through generation. Without content-level input/output evidence, generated propositions cannot be classified uniformly as faithful reshaping, entailed derivation or novel conjecture (MEM-OBJ-source-text, MEM-OBJ-wiki-pages, MEM-OBJ-document-chunks, EPI-OBJ-wiki-pages). Thus the project's compilation claim is supported as retained reuse, not as demonstrated production of accepted new knowledge (EPI-CLM-wiki-compilation).
From retained state to an answer
The application dispatches the turn; a model does not schedule independent workers. In default pre-retrieval mode, code filters history and selects evidence before invoking the answer agent. Question vocabulary, aliases, blacklist, FTS, open-page identity and collection intent choose among wiki chunks, raw sources, whole overview/index bodies and dataset context. An uncovered follow-up can use a separate condenser invocation; only its rewritten question returns to the retrieval gate. Defaults of six wiki chunks or four raw chunks are count limits, not a global token budget. History removes refused exchanges, compacts advisory tables and keeps a six-message tail. Removed conversation text has no automatic reload pointer; querying current data again does not recover past conversational values (RT-RTE-preretrieval-chat, memory annotations).
Grounding has different force at different return boundaries:
- Strict: PydanticAI owns the model/tool loop. The service substitutes refusal unless its inspected messages contain a substantive tool return. This predicate does not compare answer meaning with evidence; a retrieved result can be ignored and still satisfy it. Supplied-history semantics also qualify the predicate's scope (RT-RTE-strict-chat).
- Pre-retrieval: Scope and context selection can prevent generation. Post-processing tests tool presence only when no context was injected, and applies lexical coverage to raw-source answers, not equivalently to curated context. Open-page, collection and dataset/advisory alternatives therefore matter to any grounding claim (RT-RTE-preretrieval-chat).
- Streaming: Reachable web and marimo dispatch emits text immediately without strict final grounding substitution. This establishes an unchecked output route, not an observed incorrect answer (RT-RTE-streaming-chat).
Standard chat grants retrieval, not wiki writing. Separate services perform ingestion, saving and maintenance. This is a composition-time tool boundary, not an OS sandbox: the generic factory permits caller-supplied Python tools, and other application paths can invoke host subprocesses. Configured models are not immutable weight identities; requested sampling settings do not establish deterministic output (RT-CLM-readonly-chat, RT-RTE-configure-grants, RT-CMP-openai-model, runtime control envelope).
Structured datasets add a distinct route. The finance overlay validates required metrics and attributes, admits eligible currency/amount/term combinations, computes conditional gains in Python and ranks results. Non-deterministic alternatives are separated from ranked projections. This supports arithmetic under encoded premises and a constant-rate model, not authenticated quotations or future-return predictions. A postprocessor can append the computed table, but only when its marker test does not find an existing table marker; it does not verify that an already-present model table matches (MEM-OBJ-datasets, EPI-RTE-finance-check, EPI-RTE-finance-select, EPI-RTE-finance-derive, EPI-RTE-answer-format).
From an answer back into retained knowledge
The representative workflow crosses the important admission boundary: web question → displayed answer → separately requested concept draft → editable review → save. Drafting selects answered exchanges, omits refusals and unanswered questions, and merges an existing same-slug page when present. The human can edit, verify or cancel. Optional draft verification compares submitted editor text with passages retained from that conversation, without re-retrieving sources. A successful save return displays a notice and refreshes navigation; a draft or displayed answer alone is not a retained wiki page (workflow account, RT-RTE-chat-save).
Review is a normal web protocol, not an enforced server provenance condition. The save endpoint requires nonempty input and an available lock, not a prior draft or verification pass. Marimo and convenience-service paths structure and write without intermediate review. Web saving avoids another generation call, but deterministic repairs can still change the resulting page. Git history is attempted rather than required for delivery. On failure the dialog preserves editor contents for human retry; it does not roll back already-written files or derived state. Ingestion compensation cannot be assumed for this different save sequence (RT-RTE-chat-save, workflow save annotation).
The strongest supported learning-related contribution is this software-admitted conversation-to-page chain: recorded conversational text becomes a durable page that later retrieval or merging can consume. It establishes a wired trace-fed update on marimo/convenience paths, independently of improved capacity or factual acceptance. Human-reviewed web admission is a different mechanism. Raw trace files, reduced turn history and a saved page are not interchangeable artifacts. The accepted profile verification confirms this local positive while preserving incomplete trace-source and update coverage (MEM-OBJ-wiki-pages, MEM-OBJ-spans, profile verification).
Criticism, maintenance and the assessment it permits
Optional NLI verification scores selected checkable sentences against selected passages. Page results retain probabilities, hashes and abbreviated excerpts; current thresholds can change later classifications without new inference. These results inform readers and lint, not a universal admission or retrieval veto. Answer annotation follows display. Fresh hashes license reuse of a diagnostic, not endorsement of source assertions (EPI-RTE-verify-check, MEM-OBJ-page-verifications, EPI-RTE-verify-use).
Maintenance preserves some explicit criticism. A shared-source, truncated page comparison can produce a possible-contradiction warning; repair retains it rather than adjudicating which claim to reject. Gap reminders can later become links on lexical coverage evidence, not a demonstrated semantic resolution. Source withdrawal removes raw access and one-to-one summaries but leaves other citing concepts stale and searchable. History restoration is conditional; a usable database snapshot preserves state that file-based reindex cannot reconstruct (EPI-RTE-contradiction-check, EPI-RTE-contradiction-note, EPI-RTE-gap-note, MEM-RTE-source-withdrawal, MEM-OBJ-index-snapshots).
For personal source-backed browsing and reusable summaries, the implementation establishes acquisition, retention, multiple read-back selectors and explicit maintenance effects. For the stronger scenario of accepting new factual conclusions and improving future action through criticism, the evidence stops earlier. Retained warnings and update routes are a contribution, but no supplied chain establishes theory-builder condition 1's localized formulated theory, condition 2's content-dependent consumption, condition 3's blame-assigned criticism with revision or changed reliance, and condition 4's reuse of that result as separately satisfied properties. Nor is there evidence that criticism of a consumed theory improved future capacity. These are unestablished conclusions, not evidenced absences or consequences of fixed model weights (epistemic bounded conclusion, RT-RTE-wiki-maintenance).
Reflection, autonomy and self-improvement remain independent questions. The records do not establish the two causal directions needed for reflection or computational performance of every internal theory-building role. They do establish automatic substeps, despite human initiation. Retained pages and controls can affect later behavior, but no supplied evidence closes an exercised self-improvement pathway from objective-bearing evidence through update determination and live adoption to subsequent operation causally depending on the change. No controlled comparison demonstrates successful improvement (RT-RTE-configure-grants, RT-RTE-chat-save, epistemic bounded conclusion).
Linked before/after content and execution traces could establish particular transformations, exercised consumption and responses to criticism. A recorded evidence-consuming acceptance criterion, intended use and scope would change the acceptance assessment. A comparison independently varying a mechanism would be needed for its causal benefit. Likewise, enforcing draft provenance or a semantic save gate would change the current admission guarantee; excluding stale derivatives from retrieval would change withdrawal's force. Existing evaluator code alone does none of these. Fixed suitability probes, human-mediated packets and frozen regression fixtures support bounded checks, while the report verification explicitly retains incomplete evaluator coverage (RT-RTE-model-evaluation, EPI-RTE-eval-packet, EPI-RTE-corpus-freeze, report verification limits).
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No observed workload or causal comparison | SRC-1; RT-RTE-document-ingest, RT-RTE-chat-save, RT-RTE-model-evaluation | Frozen implementation/design; no target, provider or tests executed | Successful delivery, reliable compliance, beneficial activation and causal improvement are not established | Provenance-linked execution artifacts; objective-based outcome comparisons with independently varied mechanisms for component attribution |
| Source truth and model-mediated semantic relations remain unresolved | SRC-1; MEM-OBJ-source-text, EPI-OBJ-wiki-pages, MEM-OBJ-datasets, EPI-RTE-finance-derive | Imported documents/data and inspected generation/formula integrations; provider internals excluded | Neither accepted novel knowledge, uniformly faithful compilation nor actual financial returns follow | Authenticated premises, linked input/output evidence and explicit evidence-consuming acceptance decisions; validated formula encoding and assumptions |
| Evaluator inventory is partial; the battery additionally retains service-driven sentence-verification results | SRC-1; RT-RTE-model-evaluation, EPI-RTE-eval-packet, EPI-RTE-corpus-freeze, epistemic coverage, report verification limits | Selected evaluator bodies; verifier's inspection of scripts/verify_battery.py; remaining evaluator consumers incompletely assessed |
No complete evaluator-inventory comparison or system-wide absence of further evaluation/feedback mechanisms | Complete consequential evaluator and retained-result consumer tracing, including the battery and feedback alternatives |
| Cross-caller refresh is not established | SRC-1; RT-RTE-configure-grants, MEM-OBJ-aliases-generated, report verification limits | Known alias triggers, no-candidate early return, cached agents and selected web resets | Retained updates do not imply immediate activation or refresh parity across interfaces | Caller-by-caller invalidation/reload tracing and linked activation evidence after each update mechanism |
| Recovery fidelity and crash behavior are incomplete comparisons | SRC-1; MEM-OBJ-index-snapshots, MEM-OBJ-document-chunks, RT-RTE-chat-save, report verification limits | Snapshot/reindex dispatch, documented metadata losses, ingestion compensation and save sequencing; not every failure audited | Universally lossless restoration, all-artifact transactions and crash consistency cannot be inferred | Full recovery/failure-path assessment and failure-injection evidence across files, database, references and history |
| Verification and maintenance coverage is local | SRC-1; EPI-RTE-verify-check, EPI-RTE-contradiction-check, EPI-RTE-gap-note, RT-RTE-wiki-maintenance | Selected passages/sentences, shared-source truncated pairs, lexical gap checks and selected repair handlers; remaining parser/extractor/repair internals not fully audited | Exhaustive source support, calibrated diagnostics, comprehensive conflict detection or semantic gap resolution are not established | Selection/encoding audits, scoped classifier evaluation and recorded criticism-to-revision outcomes; remaining consequential handler inspection |
| Initial provenance, persistence horizons and update alternatives remain partly classified | SRC-1; MEM-OBJ-datasets, MEM-OBJ-spans, RT-RTE-preretrieval-chat, RT-RTE-chat-save, profile verification | Known later transformations and local conversation-to-page witness; initial workspace admission, cross-session UI persistence and other trace-fed alternatives unresolved | No complete lineage, authority, curation, selection or trace-learning/source inventory; no system-wide trace-learning negative | Initialization/admission provenance, persistence and later-consumer traces, and inspection of remaining included alternatives |