post-graph-rag report verification

Type: agentic-system-analyses/types/agentic-system-verification.md

Verification

The reports support a bounded, implementation-grounded account. I read all supplied members, their types, the record contract, the empty correction-answer files and the structural report-check, which reports none. I independently inspected SRC-1 rather than treating the analysts' checks as verification. The checkout is clean and HEAD equals the reviewed revision. The tracked tree agrees with boundary: shipped callers and evaluators belong to the knowledge-system boundary; provider and dependency internals do not. No target code, test, example or service was executed.

Admission and model components. I checked every runtime component declaration against SRC-1 post_graph_rag/config.py and post_graph_rag/llm.py, including role selection, mutable model names, shared completion fallbacks, structured-to-plain retry, embedding dimensions, local transformer alternatives, term hashing and stream opening. The separation of wired behavior from unobserved operation is justified. I traced RT-RTE-document-admission through preparation, batch failures, passage insertion, structural-write catches and replacement in post_graph_rag/engine.py. Passage-first does not mean preparation cannot fail or replacement is atomic. I separately checked the supported external result shapes, replace/merge branches and optional validation in post_graph_rag/extractor.py; RT-RTE-extractor-extension correctly bounds the default gates. These are reachable configured interfaces, not bypasses inferred merely from unchecked types.

Retained objects and write semantics. I checked all memory object declarations against their writers and consumers in SRC-1 post_graph_rag/graph_store.py, post_graph_rag/engine.py and post_graph_rag/reporting.py: passage text, entity descriptions, identity/access fields, relation assertions and provenance, vectors, report prose and ranking metadata, and retrieval events. In particular, alias lookup and longer-description selection do not verify identity; source counts and maximum confidence do not establish corroboration. MEM-OBJ-relation-metadata and EPI-RTE-relation-upsert faithfully describe the same-key merge, polarity AND, incoming-or-prior periods and preservation of supersession fields. The numerical outputs are distinct from their model producers; the reports do not mistake storing vectors for parameter training.

Withdrawal, temporal scope and alternative channels. I followed deletion, source withdrawal, endpoint dormancy, reason-specific revival and sweep in SRC-1 post_graph_rag/graph_store.py. MEM-RTE-document-withdrawal correctly distinguishes deleted passages from surviving audit residue. I also traced entity seeds, one-hop and multi-hop walks, independently vector/lexically retrieved edges, global relations, documents and reports through the actual query consumer. RT-RTE-question-answer does not promise universal historical answering or all-channel archival exclusion: engine relation filtering still applies after the one-hop shortcut, while reports and entity descriptions lack the same temporal gates. Endpoint hydration does not inherit entity-seed archive filtering. The reports preserve those countercases instead of generalizing a guarded branch.

Reports, context and adaptation. I checked clearing and rebuilding communities, clipped member projections, parent pseudo-graphs, parent rating and later report ranking. EPI-RTE-community-render correctly identifies omitted separate date fields; its small deterministic witness follows the renderer without claiming an observed model error. RT-RTE-community-build distinguishes report usability from grounding. I followed raw-context precedence, bypass, history insertion, character-based clipping and abbreviated terminal triples in post_graph_rag/engine.py. The request object and both runtime authority-path declarations accurately distinguish symbolic selection from interpretive model guidance. I independently traced events to retention statistics, fixed scoring, non-dry archive writes and later seed/snapshot exclusion in post_graph_rag/retention.py and post_graph_rag/graph_store.py. MEM-BAP-access-state and RT-RTE-retention-control support exposure-fed adaptation, not demonstrated improved capacity.

Epistemic records and public claims. I checked every epistemic object, route and claim against its cited implementation or doctrine, including the contradiction candidate projection, offered-ID filter, declarative precedence and withdrawal consumer. EPI-RTE-contradiction-check is a consequential content-sensitive check, not source-truth certification. Its separation from disposition is sound. The indeterminate semantic classifications do not assume conjecture merely from model generation. I read the three selected scorer implementations: native element scoring and its lexical unanswerable branch, strict numeric/period-or-cosine scoring, and type-specific LongMemEval regrading. EPI-RTE-benchmark-check preserves their differing targets, parsers and operational reporting force. The README passages support the attributed score and validity-effect claims, not independent benchmark validation or component attribution. No emitted finding requires accepting those reported rates as verified measurements.

Callers, workflow and release. I independently followed the complete quickstart and helper, and its indexing and answer-return path. Workflow correctly treats print/return as delivery without an answer-correctness veto or repair loop. I checked named-realm corpus production and later retrieval, conversational mapping and cleanup, and ECT-QA replay input. MEM-RTE-evaluation-memory does not confuse prior questions/golds with authoritative answer memory; its warning about the truthy indexing dictionary is justified. The incremental example's direct chunk calls do not acquire replacement semantics. The delta implementation lacks an upper watermark predicate and deletion journal, supporting MEM-RTE-change-polling. The capped drill-down supports the afforded inspection route without establishing a deployed explorer. Packaging and the two GitHub workflows support RT-RTE-package-release, not an observed release or mandatory CI gate on publication.

Relations and integration. I checked all declared Part of: relations as executable suboperations of document admission/replacement or community building; direct API reachability does not erase their use within those containers. I checked every supersession in reconciliation: the duplicate passage, report-prose and withdrawal-route identities are supported. Entity and relation containers remain valid alongside memory's separate operative parts; community rating is narrower than the metadata container. The other route connections do not incorrectly equate checks with dispositions, producers with products, or evaluation orchestration with offline scoring. Every memory integration issue has a compatible disposition. No unresolved inter-report conflict or structural failure requires routing a correction. Empty evidenced-absence kinds make no unsupported system-complete negative.

Omission search and stopping. Selection used the frozen package/caller tree and producer-to-consumer dependencies, not just quoted anchors. I investigated whether alternate admission evades default gates, alternate retrieval channels evade archival/temporal controls, report projections preserve scoped assertions, and shipped benchmark callers add behavior beyond bare API calls. The first three are already materially covered; the last leaves the local inventory limit below. Detailed diagnostic accuracy, other scoring families, result bodies and ablation designs remain outside the reports' assessed conclusions, not unavailable evidence. Further inspection is not reasonably likely to change the retained implementation account without expanding it into a benchmark-validation or exhaustive evaluator project. Search stopped with no identified material challenge lacking a disposition.

Blockers

none

Limits

• MEM-RTE-evaluation-memory does not inventory the LongMemEval caller's answer-time guidance. Beyond dated ingestion and mix/top-k selection, SRC-1 evaluation/longmemeval/run.py prepends recency-based conflict resolution, date arithmetic and session-date inference instructions to the question passed to retrieval and synthesis. This is a local omission from a partial caller account, not evidence of an automatic grade-to-policy update or a defect in the reported engine route. Comparisons attributing this harness's answer behavior solely to retained graph state or the ordinary engine prompt must be withheld; the supplied source-truth and unverified-performance conclusions remain supported.

  "- When records conflict about the same thing, the most recent " "statement is the current truth. Give the current value only; " --- evaluation/longmemeval/run.py