LLM Repo Wiki synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
A production pipeline with several admission boundaries
LLM Repo Wiki imports GitHub repositories, drafts a wiki, maintains pages after source changes, and supplies retained source and wiki content to chat and external consumers. Its defining architecture is a code-controlled acquisition and writing pipeline beside a model-controlled chat tool loop. It is not one autonomous agent supervising the entire process.
This account covers the shipped application, separate worker, prompts, persistence, review actions, retrieval, MCP server, export and measurement drivers at commit 38a439cd649105c0cd90b687fc1f8d8dd1d0986c, with an applicability cutoff of 2026-11-09. The sole source, SRC-1, supplies implementation, declared intent and reported results. No application execution, installed database, raw benchmark trace or causal experiment was supplied. Thus the analysis establishes wiring and requested behavior, not successful operation or benefit. External GitHub content, database-engine and dependency internals, providers and MCP hosts are outside the frozen implementation evidence (boundary, Source register). Independent report verification and profile verification found no remaining blockers or declared limits; that does not upgrade source evidence to observed operation.
From repository change to proposed prose
An operator admits a repository. Manual synchronization, polling and signed default-branch webhooks then converge on a queue consumed by a separate worker. Code downloads a commit snapshot, filters and secret-scans files, chunks surviving content, obtains embeddings and persists source state. Changed files can mark citing pages stale and schedule authoring. These are automatic per-file admissions after repository selection, not human approval of every imported statement. Per-file transactions permit partial persistence on failure; the route is not an atomic snapshot replacement. Detected-secret exclusion applies to the inspected ingestion paths, not every possible secret or all wiki/chat input (RT-RTE-repo-sync).
The author worker selects pages and evidence. Models propose an initial outline or placement for uncovered changes and write structured drafts, but do not control a retrieval-tool loop. Separate draft calls receive schema, project context, selected evidence and, for updates, the current page. Code caps queries, deduplicates source hits and supplies at most sixteen hits, each clipped to 6,000 characters. This isolates writing calls from the complete repository and preceding conversation; only structured results and accounting return. It is bounded model delegation by code, not tool-using subagent orchestration (RT-RTE-author-pages, workflow).
The author filters references and drops drafts with no surviving citation. New citations must match retrieved source metadata, while updates may preserve exact old citation keys without retrieving those sources again. The filter uses full chunk line bounds even when the delivered text was clipped. Consequently, it neither proves that every cited line reached the writer nor that every factual sentence has support. Removing a bad link can leave its claim text intact. This is a useful provenance filter, not semantic proof; the README itself distinguishes grounded citations from proven statements (EPI-OBJ-source-citations, EPI-CLM-wiki-grounding).
Proposal delivery, publication and searchable delivery differ
The representative workflow is repository-change maintenance through proposals, human review and indexing. Author delivery ends at proposal persistence. Individual draft failures can coexist with delivered proposals; systemic failures reach queue retries. Retries retain earlier writes and use same-SHA/existing-page checks rather than whole-run rollback. Human rejection is not automatically converted into a criticism prompt or immediate reauthoring request (RT-RTE-author-pages, workflow).
Review actions permit acceptance, rejection, direct editing and restoration. The store rejects acceptance when the proposal's parent differs from the current revision under its locked transaction. This prevents silent prose rebasing on that path. It checks compatibility, not truth, and the local human actor label does not establish authenticated identity or substantive review. Accepted-revision restoration copies body and citations into a new revision, not the complete earlier system state (RT-RTE-revision-admission).
Accepted persistence makes the current body available to page readers, but search delivery is asynchronous. Enqueue can fail after acceptance. The index worker checks currentness before remote embedding, not again inside its replacement transaction. Search can therefore return older indexed sections; hydrated wiki hits omit indexed-revision and stale metadata. A source change marks a page stale without withdrawing its accepted prose. These are meaningful distinctions between stored content, current content and evidence actually delivered to a later model (MEM-RTE-index-page, MEM-RTE-search-knowledge, MEM-OBJ-page-revisions).
Later consumers do not share one review or grounding policy
Chat reloads server-owned history, schema, project context, tree and any summary. A model chooses search, page operations and answers within an eight-step loop. Its tools operate on wiki-scoped state, without a supplied shell or arbitrary network executor. Update, move and delete require selected tool approval; create does not. Chat creation uses apply mode and can directly create an accepted page. Ordinary chat also lacks the author's citation-stripping filter. Thus neither “all model writes are proposals” nor “all answers are mechanically grounded” describes the shipped system (RT-RTE-chat-turn).
Hybrid search combines vector and full-text ranks and hydrates selected text. Stored prose, vectors and symbolic access fields have different consumers; retrieving numerical features does not train model weights. Search requests are pull operations. Automatic assembly of retained history, summary and owner context for a turn, and code-selected evidence for a writer, also supplies content without a separate consumer request for those parts. Identity matching, chronological budgets and embedding/lexical relevance act at different selection stages (MEM-OBJ-chunks, MEM-OBJ-wiki-chunks, MEM-RTE-search-knowledge, profile verification).
MCP offers scoped search, page reads, lists, freshness and sync requests, not wiki-writing tools. Compact search deduplicates and clips hits more aggressively than application search. Explicit MCP reads can return the newest proposal for a never-accepted page, labeled unreviewed; the application review page also previews proposals. Export instead emits accepted reachable Markdown. Neither already-delivered host text nor exported files receive automatic invalidation. External host reasoning and retention remain uninspected (RT-RTE-external-consumption).
Conversation folding supplies a real trace-fed update, not demonstrated improvement
The strongest supported learning-related contribution is automatic conversation folding: retained user/assistant turns and clipped tool requests/results feed a model-written summary, which persists and enters later chat instructions. This is more than log storage and does not depend on weight updates. It establishes a wired trace-fed memory route with a later consumer, but not accurate preservation or improved future capacity (MEM-OBJ-chat-messages, MEM-OBJ-chat-summary, MEM-BAP-summary-context).
Deterministic code normally triggers folding above an estimated 60,000 tokens and retains about 20,000 recent tokens in whole user-turn groups, always preserving the newest group. Tool inputs/outputs are clipped to 300/400 characters before summarization; the summary call has a 1,200-token output cap. Old rows remain stored, but the chat tools provide no arbitrary history-reload operation. These are context-selection controls, not a hard total-window guarantee. Regeneration truncates messages without repairing summary/cutoff, so derived memory can retain information from removed turns (RT-RTE-context-fold).
Editable schema and retained owner brief separately shape author/chat instructions. The schema is reread per call; the summary is automatically derived experience. Configurability, persistence and instruction placement do not establish reliable model compliance or an evidence-responsive criticism loop (RT-BAP-schema-instructions, MEM-BAP-project-brief).
The accepted records do not establish a complete theory-builder conditions 1–4 assessment: localized editable guidance is present, but delivery alone does not prove decisions depend on its content, and content-directed criticism with retained results shaping another round is unestablished. Criticism of a consumed theory improving future capacity is independently unestablished. Reflective operation is not established by editable guidance or summaries alone. Whole-workflow autonomy is not established: human-facing review owns a material admission decision, though several computational stages run automatically. Exercised self-improvement at this boundary is also unestablished: a durable summary has a live delivery path, but subsequent behavior causally depending on an objective-directed organizational change is not shown. These are separate evidence limits, not one negative grade (RT-RTE-author-pages, RT-RTE-revision-admission, RT-RTE-context-fold, epistemic conclusion).
Checks support different conclusions
Daily lint derives structural findings, not semantic contradictions or automatic repairs. Retrieval evaluation scores expected file paths, not answer truth. Author/chat/MCP benchmarks respectively count output/reference properties, expected citation paths and named-file recall with token cost. Some measurement routes mutate production state: author benchmarking wipes all pages of the selected wiki, while chat benchmarking exposes apply-mode tools. Evaluation is not uniformly read-only (MEM-RTE-lint-wiki, RT-RTE-retrieval-eval, EPI-RTE-benchmark-checks).
An offline citation evaluator does request semantic assessment: independent model judges compare extracted cited sentences with cited lines under explicit support criteria. It retains verdicts and reports, and can resume from prior results. This is stronger than reference membership checking, but its verdicts do not automatically admit, withdraw or revise production pages. Quote occurrence does not veto an unsupported model verdict, synthetic nonoverlapping citations are imperfect negatives, and uncited claims fall outside extraction. The reported 87% supported result is conditional on 215 agreed judgments, not all wiki sentences or an independently verified execution (EPI-RTE-citation-evaluation, EPI-CLM-evaluation-results).
For repository-change drafting, the supported result is a reviewable, source-linked proposal pipeline with conflict checks and partial-failure recovery. For factually reliable answering, the unresolved questions are semantic support, delivered evidence freshness and model compliance. Concrete evidence that would alter this assessment includes linked sync-to-review-to-search traces, sentence-level source judgments with independent labels and complete coverage accounting, and later-turn comparisons tracing summary changes to behavior. Atomic revision-aware indexing or a production semantic-verdict consumer would change the corresponding architectural conclusions; neither should be inferred from current labels or offline scores.
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No supplied execution, deployed content or causal comparison | SRC-1; RT-RTE-author-pages, EPI-CLM-evaluation-results | Frozen shipped implementation and attributed reports, not a live installation | Successful end-to-end operation, actual human review, measured accuracy, performance or benefit | Linked execution artifacts, retained inputs/outputs and scoped outcome comparisons |
| Reference filtering does not decide semantic support | SRC-1; EPI-OBJ-source-citations, RT-RTE-chat-turn | Author grounder, preserved keys, clipped evidence and ordinary chat | Every claim is supported, every cited line was delivered, or every consumer shares the same grounding gate | Actual drafts with delivered evidence and independently checked claim-to-source relations; route-specific enforcement changes for stronger guarantees |
| Current page and delivered search text can diverge | SRC-1; MEM-RTE-index-page, MEM-RTE-search-knowledge | Acceptance enqueue, pre-embedding guard, replacement transaction and hydration | Immediate or strictly synchronized search freshness; measured race frequency | Interleaving traces and indexed/current revision comparisons; revision-aware transactional enforcement for an invariant |
| Summary fidelity and derived rollback are unchecked | SRC-1; RT-RTE-context-fold, MEM-BAP-summary-context | Clipped transcript, persisted summary/cutoff and regeneration | Faithful compression, complete rollback or improved future capacity | Transcript-summary audits, regeneration traces and controlled later-turn comparisons; coordinated summary rollback |
| Initial content provenance and generic caller control remain unknown | SRC-1; MEM-OBJ-project-context, MEM-OBJ-page-revisions | Known subsequent write/read paths without installed database history | Complete lineage or universal per-content human admission | Installed-state provenance and recorded admission decisions for the unresolved mechanisms |
| Provider, SDK and host guarantees are external contracts | SRC-1; RT-CLM-provider-routing, RT-CMP-chat-model, RT-RTE-external-consumption | Request preferences, model identifiers, approval wiring and delivered MCP text | Actual remote deletion, exact immutable weights, independently verified SDK enforcement or downstream host behavior | Pinned dependency/provider evidence and scoped operational audits; host traces for downstream claims |
| Evaluator targets, subsets and downstream force differ | SRC-1; EPI-RTE-citation-evaluation, EPI-RTE-benchmark-checks, EPI-CLM-evaluation-results | Offline sentence judges, path proxies, whole-arm comparisons and reported agreed subset | Unconditional factual accuracy, isolated component effects or automatic production acceptance | Complete raw judgments with independent labels, coverage/error accounting, component interventions and an explicit production disposition consumer |
| Trace-fed memory does not settle improvement or complete learning inventory | SRC-1; MEM-OBJ-chat-summary, MEM-OBJ-wiki-log, MEM-OBJ-eval-results | Wired summary update plus diagnostic retention/resume; no exhaustive learning absence warrant | Demonstrated learning, exercised self-improvement or exhaustive absence of other learning routes | Objective-linked update and later-consumer traces with causal dependence; bounded inspection of unresolved diagnostic alternatives |