HippoRAG synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From documents to retained search state
HippoRAG is a document-memory system, not the enclosing agent that decides which arbitrary tools to use. This account covers its shipped Python library and callers, optional Model Context Protocol (MCP) service, plugin and maintenance skills at commit d5c8329422e0a0b834a15874545cb6a74b4f9b26, with an applicability cutoff of 2026-10-16. The evidence is inspected implementation and attributed doctrine/design from SRC-1, the frozen HippoRAG repository. There is no registered execution trace or causal experiment. The whole-system functional boundary includes shipped maintenance and consumption paths; it does not imply exhaustive inspection of every adapter or external service. See boundary and Source register.
The strongest supported contribution is a wired path from new documents to durable, later-searchable state, including a shipped caller that reopens an index before consuming it. Original passages remain separate from the model-derived representation that selects them. This establishes a cross-instance memory mechanism, not observed successful reuse or improved answer quality. The public claim of recognizing connections has an operational counterpart in derived graph links and retrieval, without establishing reliable recognition or a biological equivalence: RT-RTE-index-state, MEM-OBJ-passage-text, EPI-CLM-memory-connections.
A caller supplies strings or provenance-bearing chunks. Default preprocessing leaves each document intact. Online extraction performs named-entity recognition and then relation-triple generation. Code parses and filters structures, normalizes strings, encodes passages/entities/facts and constructs source-aware graph edges, including similarity-based entity links. The extraction, QA and embedding roles can use configured services or injected components; the inspected orchestration does not update their parameters, but it also does not enforce immutable weight identities. See EPI-RTE-openie-generation, RT-CMP-extraction-llm, RT-CMP-qa-llm, RT-CMP-embedding-model.
Structural admission is not acceptance of relation truth. Parsing, triple arity and duplicate removal permit indexing without testing whether a relation follows from its passage. Normalization can also collapse distinctions: the recorded implementation deduction maps both C++ and C# to c. Original passages survive, so this is a risk to search identity rather than universal destruction of source meaning. The semantic relation between actual extraction outputs and inputs remains indeterminate without paired evidence: EPI-RTE-triple-check, EPI-RTE-triple-admission, EPI-RTE-text-normalization.
Reuse, refresh and withdrawal
Default persistence combines Parquet text/vector stores, JSON extraction/provenance state and a pickled graph. The extraction state retains source passages and extracted structures, not the full response metadata or a justification for each relation. A manifest and preparation checks can veto incompatible recorded configurations or inconsistent store/graph identities. These are bounded compatibility checks, not truth certificates or a transaction across all writes: passage persistence precedes extraction and later graph construction. See MEM-OBJ-openie-state, MEM-OBJ-graph-state, RT-CLM-state-compatibility.
Passage identity is a content hash; source_id is current caller-supplied metadata. Repeated identical text can replace that metadata without new extraction or embedding. Two sources containing identical text therefore do not acquire independent ownership of the same passage. Deletion removes selected source contributions and unreferenced derivatives while retaining facts supported by surviving chunks. Refresh by delete then index is not atomic replacement. Separately, the inspected SQLite response cache can replay raw model outputs for matching requests and is not erased by document deletion. Withdrawal from the searchable index is consequently not secure forgetting: MEM-OBJ-chunk-metadata, RT-RTE-index-state, MEM-RTE-response-reuse.
From a question to an answer
On the ordinary route, query embeddings select candidate facts, a generative model judges their relevance, and code remaps generated selections to candidate strings. Candidate membership is constrained, but nearest-string matching with cutoff zero does not guarantee faithful model intent, relevance or truth. Selected facts seed personalized PageRank over an undirected graph projection alongside dense passage weights. Empty or failed filtering instead selects dense retrieval. The library, not a general model dispatcher, owns this progression: EPI-RTE-fact-check, EPI-RTE-fact-selection, RT-RTE-retrieve-answer.
QA receives selected original passage text, the question and static guidance—not the whole graph, source metadata or extraction rationale. Context selection is primarily count-bounded, not an aggregate token guarantee. The memory account records defaults of five linked candidates, 200 retrieved passages and five QA passages; default preprocessing itself imposes no passage splitting. Material outside the prompt remains reachable through further retrieval, not through dereferenceable pointers in the model window. This is requested retrieval, not independently established unsolicited memory injection. See memory core ideas, MEM-OBJ-passage-text, RT-RTE-retrieve-answer, profile verification.
The returned answer is model text, with delimiter-based parsing rather than a semantic correctness gate. “Grounded” therefore denotes passage-conditioned generation here, not demonstrated entailment. Optional benchmark evaluation compares answers to supplied aliases using normalized exact match and token F1, and passages to exact document strings using recall. Scores are logged or returned; the inspected driver does not use them to accept claims or revise the pipeline. Host guidance asks an agent to check server answers against passages, but actual checking is unobserved: EPI-CLM-grounded-answer, EPI-RTE-metric-evaluation, EPI-RTE-host-check.
Material alternatives qualify this map. Direct dense retrieval and the separate StandardRAG baseline bypass graph filtering; supplied query solutions can bypass retrieval. IRCoT uses generated thoughts as further queries, merges passage scores and stops on an answer marker or step limit. Those thoughts are current-call state, not demonstrated durable method revision. vLLM offline extraction saves state then deliberately requires a later online build; Transformers-offline participates in indexing and has different parsing/error behavior. Neither inherits every online rejection guarantee. See RT-RTE-standard-rag, RT-OBJ-query-solution, RT-RTE-ircot-retrieval, RT-RTE-offline-extraction.
Host-directed folder ingestion
The representative multi-step workflow is the shipped folder-indexing skill. The host selects files, excludes unsuitable material, prepares roughly 100–400-word passages, presents a plan and seeks confirmation above about 50 passages. Refresh first deletes by source ID; batches then submit prepared texts and provenance, and the host is instructed to report counts and skipped-file reasons. This is a connected prescribed workflow, not an observed host session. Confirmation and chunking are host-interpreted policy, not server-enforced approval or token limits. The skill's one-extraction-call/no-retrieval-LLM guidance is not universal: online extraction has two stages and ordinary filtering is model-backed. See WF-RTE-folder-indexing, EPI-CLM-call-guidance.
The MCP service serializes individual operations on one instance, returns typed counts/results and converts failures into tool errors. The host retains responsibility for subsequent batches, recovery and final reporting. Counts are not per-file or semantic audits; a successful delete followed by failed replacement can leave old content unavailable. Read-only launch omits index/delete tool registration, but does not prevent every filesystem/cache write or coordinate independent processes. See RT-RTE-mcp-tools, WF-RTE-folder-indexing.
What the evidence changes—and does not
For a scenario requiring persistent document acquisition and later passage selection, the supported result is a concrete implementation with reload consumers, explicit withdrawal and alternate retrieval routes. For a scenario requiring verified factual admission, atomic refresh, secure erasure or demonstrated answer improvement, those mechanisms do not establish the required conclusion. No product ranking follows from this distinction.
Durable document additions can change what later retrieval can access. Whether this improves capacity for future action remains unestablished; no criticism-to-revision-to-later-benefit comparison is recorded. In particular, criticism of a consumed theory is not shown to improve future capacity. The runtime supplies no formulated-theory assessment establishing theory-builder conditions 1–4, and these conditions cannot be inferred from triples, relevance judgments or benchmark scores. Reflection is independently unestablished: retained source/configuration state does not by itself evidence both causal directions of a self-representation. Internal library sequencing is wired, but autonomous theory building and autonomous ownership of the host-directed workflow are not established. See RT-RTE-index-state, EPI-RTE-metric-evaluation, epistemic bounded conclusion, WF-RTE-folder-indexing.
The same positive memory contribution does not establish self-improvement at this boundary. No objective-bearing evaluation is shown determining a durable method update that later operation exercises; neither retaining new documents nor loading static prompts closes that pathway. This is not a system-wide absence claim. Trace-fed learning also remains unresolved because arbitrary admitted documents lack a recorded trace-origin inventory, raw response replay alone establishes no qualifying derived update, and alternative caches remain uninspected. See RT-RTE-index-state, MEM-RTE-response-reuse, profile verification.
Paired passage/extraction/answer traces with semantic adjudication would change the factual-fidelity assessment. Controlled comparisons varying the graph/filter or a retained revision would change the benefit and attribution assessment. A recorded evaluation-driven update with a later consuming execution would be needed to establish the currently missing self-improvement links. Crash interventions, cache inspection and multi-process traces would separately change recovery, erasure and concurrency conclusions. These are different missing warrants, not one epistemic grade. No target-source reads were added for this synthesis.
Limitations
Both handed verifications declare no blockers and no additional limits; reconciliation supplies no unresolved conflict. The following evidence and system limits remain consequential to the bounded account, rather than defects waived by verification. See report verification and profile verification.
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| No observed execution or causal comparison | SRC-1; RT-RTE-index-state, RT-RTE-retrieve-answer | Frozen implementation, instructions and executable callers; no target execution | Successful reuse, measured accuracy/cost, comparative benefit and causal component effects | Registered execution traces and appropriately controlled comparisons |
| Structural admission and relevance do not check truth | SRC-1; EPI-RTE-triple-admission, EPI-RTE-fact-selection, EPI-RTE-text-normalization | Online extraction, normalization and ordinary filtering; alternate/injected behavior unresolved | Faithful extraction, relevance guarantees or epistemically accepted generated knowledge | Paired inputs/outputs, source-entailment checks and recorded consequential semantic dispositions |
| Compatibility is neither immutable producer identity nor a transaction | SRC-1; RT-CLM-state-compatibility, RT-CMP-embedding-model | Checked manifests, graph/store/OpenIE identities and ordered writes | Exact weight reproducibility, complete rollback or crash-safe refresh | Pinned producer identities, verified transaction/recovery mechanism and interrupted-write tests |
| Content-hash identity does not retain independent identical-text ownership | SRC-1; MEM-OBJ-chunk-metadata | Current metadata replacement and source-ID deletion targeting | Independent source withdrawal for identical passage text | Multi-owner provenance with traced deletion semantics |
| Index deletion leaves separately cached output | SRC-1; MEM-RTE-response-reuse | CacheOpenAI and core delete; other provider caches uninspected | Secure forgetting or a complete cache-erasure comparison | Inventory of all retained copies and verified coupled erasure |
| Host policy and local queue have limited enforcement | SRC-1; WF-RTE-folder-indexing, RT-RTE-mcp-tools, EPI-RTE-host-check | Skills, one service instance, tool registration; host execution and deployment permissions opaque | Exercised approval/checking, completed folder delivery, global read-only isolation or cross-process exclusivity | Host session traces, deployed permission/authentication evidence and multi-process coordination tests |
| Alternative providers, stores and input origins remain incompletely classified | SRC-1; MEM-OBJ-passage-text, MEM-RTE-response-reuse, profile verification | Default objects and named routes; alternate caches/injections and external backend internals unresolved | Complete storage/authority/read-back enumeration, pull-only whole-boundary status or positive/negative trace-learning classification | Accepted records tracing alternative retained parts, admission, original inputs and later consumers |
| Benchmark agreement has a restricted oracle domain | SRC-1; EPI-RTE-metric-evaluation | Implemented normalized alias/token and exact-document comparisons; bulk datasets/fixtures uninspected | Source truth, reasoning validity, dataset quality or reproduced benchmark performance | Inspected reference provenance, semantic adjudication and registered reproduction evidence |
| Future-capacity improvement and self-change pathway are unestablished | SRC-1; RT-RTE-index-state, RT-RTE-ircot-retrieval, EPI-RTE-metric-evaluation | Durable document/search writes, current-call thoughts and reported scores; no recorded evaluation-driven method revision | Criticism-based learning, theory-builder conditions, reflection, autonomous theory building or exercised self-improvement | Separate property-specific records, including content-directed criticism, retained consequences and later consuming operation; controlled evidence for beneficial attribution |
| Launcher does not freeze a deployed revision | SRC-1; boundary, RT-BAP-plugin-skills | Frozen plugin specification references mutable Git installation; external host/providers excluded | Applicability to arbitrary later installations or current remote services | Immutable deployed revision and a new matching evidence boundary |