sqlite-graph-memory epistemic report
Type: agentic-system-analyses/types/agentic-system-epistemic-report.md
Source-and-claim boundary
The question is what content this shipped memory layer acquires, transforms, checks and licenses for later reliance. The functional scope and exclusions are those of the boundary; evidence identities are in its Source register, SRC-1. The private home pipeline, external agent reasoning and editing, model weights and host compliance are not inside the frozen executable account. No target execution was performed. Implementation establishes capacity and wiring, not activation or benefit.
The ordinary retrieval path is not an answer author. It imports note text, selects indexed snippets and returns context. Its numerical relevance scores are not truth evaluations. The shipped benchmark adds an answer selector, abstention and reference-based checking; these must not be projected onto MCP. Temporal closure instructions request a stronger reliance discipline than the retriever enforces. Relevant claims are EPI-CLM-search-layer, EPI-CLM-measurement-scope, RT-CLM-temporal-scope and RT-CLM-scoped-return.
Coverage
Direct reads below supply implementation unless explicitly marked doctrine. Supplied runtime findings remain distinguishable from direct inspection. All paths are SRC-1 at the reviewed commit; the source register is not copied here.
| Entry point or operation | Source path | Coverage / records or limit |
|---|---|---|
| CLI registration; MCP tool registry and dispatch, including direct-test wrapper | pyproject.toml; src/sqlite_graph_memory/mcp_server.py |
Direct, complete reads; RT-RTE-mcp-recall, RT-CLM-scoped-return. Only recall is advertised; error handling checks successful transport, not source truth. |
| Index admission, refresh, dropped files, clipping | src/sqlite_graph_memory/index_notes.py |
Direct, complete; RT-RTE-index-admission, EPI-OBJ-note-snippets. |
| Dense, graph, entity, supersession, PPR call sites, A/B, formatted delivery | src/sqlite_graph_memory/brain_ask.py |
Direct, complete; RT-RTE-recall-options, RT-BAP-context-delivery. PPR internals and backend internals are supplied runtime findings, not independently reread. |
| Encoder and reranker versions, mutability | src/sqlite_graph_memory/onnx_backend.py; src/sqlite_graph_memory/ppr.py |
Supplied RT-CMP-e5-model, RT-CMP-rerank-model, RT-RTE-recall-options. Exact weights and model internals unavailable; prevents semantic guarantees from numerical scores. |
| Stop registration, capture and ledger inspection | examples/claude-code-stop-hook.json; src/sqlite_graph_memory/turnstate_hook.py; src/sqlite_graph_memory/turnstate_show.py |
Direct, complete; RT-RTE-turnstate-capture, EPI-OBJ-turns. Viewer is a consequential human-facing consumer, not automatic recall integration. |
| Gold construction; scoring, persistence, mutant withdrawal | eval/build_gold.py; eval/run_eval.py |
Direct, complete; RT-RTE-evaluation-paths, EPI-OBJ-evaluation-records. run_ablation.py composition supplied by runtime, not independently reread. |
| Benchmark answer and abstention | bench/adapters/sgm.py; bench/adapters/_extract.py; bench/vault/tool-lanternfish.md |
Direct, complete; EPI-RTE-extract-answer, EPI-RTE-answer-abstain. Concrete static deduction below, not execution. |
| Benchmark scoring, result gate, rerun comparison and CI consumer | bench/bench_core.py; bench/validate.py; bench/run.py; .github/workflows/bench.yml |
Direct, complete; EPI-RTE-bench-score, EPI-RTE-bench-gate. Result artifacts, generator, questions and leaderboard implementation not assessed for observed outcomes or causal gains. Other-system adapters are not characterized. |
| Diagnostic checks and verdicts | src/sqlite_graph_memory/vault_stats.py; src/sqlite_graph_memory/doctor.py |
Direct, complete; EPI-RTE-diagnostic-checks. These were not assessed in detail by runtime; they add bounded truth-apt graph/environment reports, not note validation. |
| Temporal writer/reader rules and public claims | BITEMPORAL.md; README.md |
Direct, complete, doctrine/design and attributed reported operation; RT-BAP-temporal-instructions, claim records below. |
| Product change and publishing | AGENTS.md; .github/workflows/tests.yml; .github/workflows/publish.yml; docs/RELEASING.md |
Supplied RT-RTE-change-admission. Repository protection, actual maintainer decisions and deployed successor use uninspected. |
| Remaining included support, tests, paper, alternate agent guides, temporal design note | Paths inventoried in boundary | Available but not independently inspected here. No claim of a system-complete absence of knowledge production, a complete doctrine census, passing tests or private-system performance follows. The negative conclusions below concern the enumerated executable routes only. |
Missing host traces prevent determining whether context changed an agent's decisions or ledger text reached an agent. Missing authenticated execution and comparison evidence prevent assigning observed operation or causal benefit to the reconstructed routes. Uninspected benchmark result conditions prevent adopting the README's numbers as verified outcomes. Missing note-author evidence prevents assigning imported propositions a source warrant; missing provider internals prevents individuating truth-apt propositions in distributed numerical state. None of these limitations erases the inspected architectural capacities.
Epistemic-object inventory
| Object ID | Candidate truth-apt content or none | Claimed role | Evidence source ID and local anchor | Gap/limit |
|---|---|---|---|---|
| EPI-OBJ-note-snippets | Imported note assertions, potentially knowledge or instructions; stored substrings, not guaranteed whole assertions | see record | SRC-1, src/sqlite_graph_memory/index_notes.py |
External authorship and truth unknown; frontmatter and later qualifications can be omitted. |
| EPI-OBJ-recall-state | Scores/edges express selection relations; no individuated world-knowledge proposition in vectors | see record | SRC-1, src/sqlite_graph_memory/brain_ask.py |
Numerical state is not a truth certificate. Current graph and old snippets can have different freshness. |
| EPI-OBJ-turns | Text of asks, assistant statements and proposed decisions; structured claims that tools/files/commands appeared | see record | SRC-1, src/sqlite_graph_memory/turnstate_hook.py |
Tool invocation is not successful effect; asserted decisions are not validated commitments or facts. |
| EPI-OBJ-evaluation-records | Reference-note labels; extracted answers and citations; metric propositions about supplied outputs | see record | SRC-1, bench/bench_core.py |
Label fidelity and execution provenance bound all conclusions. |
| EPI-OBJ-diagnostic-report | Graph counts/date classifications and environment/file-presence observations | see record | SRC-1, src/sqlite_graph_memory/vault_stats.py |
Health thresholds are policy judgments, not evidence that a note is true or a deployment works. |
Authority-route ledger
Each ledger row is a consequential function, not an entire API. Repeated implementations are split by function. Fields owned by an existing record are cited rather than copied. All implementation rows have unobserved activation; none grants warrant beyond the indicated target domain.
| Route ID | Route function | Architectural status | Object/candidate ID | Content/update relation | Transition or check target | Evaluator/condition and domain | Activation and timing | Possible or observed result | Implemented force | Epistemic authority and scope | Operational authority: behavior permitted, blocked, or changed | Behavioral-authority path: consumer, channel, force, horizon | Evidence source ID and local anchor | Claim IDs or none | Mismatch marker or none | Gap/limit |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RT-RTE-index-admission | content transformation | implemented | EPI-OBJ-note-snippets | truth-apt transformation: acquisition/import | see route | see route | see route | see route | see route | Source warrant unknown, not established by indexing | see route | see route | SRC-1, src/sqlite_graph_memory/index_notes.py |
EPI-CLM-search-layer | none | Import precedes substring reshaping; no semantic review. |
| RT-RTE-index-admission | retention | implemented | EPI-OBJ-note-snippets | truth-apt transformation: non-ampliative reshaping | Indexed snippets/vectors | see route | see route | see route | see route | Retains selected text, not accepted claims | see route | see route | SRC-1, src/sqlite_graph_memory/index_notes.py |
none | none | Clipping preserves characters, not necessarily assertion scope. |
| RT-RTE-index-admission | lineage/freshness/recovery | implemented | EPI-OBJ-note-snippets | no content change | File change/removal and artifact reuse | see route | see route | see route | see route | Freshness only under file fingerprint assumptions | see route | see route | SRC-1, src/sqlite_graph_memory/index_notes.py |
none | none | Removal/rebuild is not epistemic rejection or rollback of an accepted theory. |
| RT-RTE-recall-options | operational admission/selection/consumption | implemented | EPI-OBJ-recall-state | no content change | Candidate membership/order | see route | see route | see route | see route | Relevance/graph proximity only; no statement truth license | see route | see route | SRC-1, src/sqlite_graph_memory/brain_ask.py |
RT-CLM-temporal-scope | Temporal rule not enforced | Supersession access is not withdrawal of stale assertions. |
| RT-RTE-mcp-recall | operational admission/selection/consumption | implemented | EPI-OBJ-note-snippets | truth-apt transformation: non-ampliative reshaping | Bundle delivery | see route | see route | see route | see route | Advisory imported excerpts, no acceptance | see route | see RT-BAP-context-delivery | SRC-1, src/sqlite_graph_memory/mcp_server.py |
EPI-CLM-search-layer | none | Child exit success is transport success, not knowledge warrant. |
| RT-RTE-turnstate-capture | content transformation | implemented | EPI-OBJ-turns | truth-apt transformation: acquisition/import | Transcript events | see route | see route | see route | see route | Original statements acquired with unknown source warrant | see route | see route | SRC-1, src/sqlite_graph_memory/turnstate_hook.py |
none | none | Structural event parsing can warrant only bounded claims about supplied events. |
| RT-RTE-turnstate-capture | retention | implemented | EPI-OBJ-turns | truth-apt transformation: non-ampliative reshaping | Selected/clipped ledger fields | see route | see route | see route | see route | Retained candidate/history, not acceptance | see route | see route | SRC-1, src/sqlite_graph_memory/turnstate_hook.py |
none | none | No checks of assistant claim truth or tool success. |
| RT-RTE-turnstate-capture | operational admission/selection/consumption | implemented | EPI-OBJ-turns | truth-apt transformation: non-ampliative reshaping | Viewer-selected rows | see route | see route | see route | see route | Human-readable operational history | see route | see route | SRC-1, src/sqlite_graph_memory/turnstate_show.py |
none | none | Viewer drops evidence pointer/offset fields and clips further; automatic model consumption unestablished. |
| EPI-RTE-extract-answer | content transformation | implemented | EPI-OBJ-evaluation-records | truth-apt transformation: non-ampliative reshaping | see route | see route | see route | see route | see route | Attribution to synthetic text only | see route | see route | SRC-1, bench/adapters/_extract.py |
EPI-CLM-measurement-scope | Not production answering | Lexical overlap is not entailment checking. |
| EPI-RTE-answer-abstain | disposition/acceptance | implemented | EPI-OBJ-evaluation-records | no content change | see route | see route | see route | see route | see route | Operational answer admission only | see route | see route | SRC-1, bench/adapters/sgm.py |
none | none | No evidence-consuming acceptance of a new ampliative claim. |
| RT-RTE-evaluation-paths | check/evidence production | implemented | EPI-OBJ-evaluation-records | truth-apt transformation: entailed derivation | Ranks versus reference sets | see route | see route | see route | see route | Metrics conditional on gold and selected composition | see route | see route | SRC-1, eval/run_eval.py |
EPI-CLM-measurement-scope | Shared functions, different policy composition | Gold generation derives labels from notes/links, not independent truth judgments. |
| RT-RTE-evaluation-paths | retention | implemented | EPI-OBJ-evaluation-records | no content change | Evaluation history | see route | see route | see route | see route | Retention of measurement evidence, not production acceptance | see route | see route | SRC-1, eval/run_eval.py |
none | none | Mutant mode explicitly withholds score persistence. |
| EPI-RTE-bench-score | check/evidence production | implemented | EPI-OBJ-evaluation-records | truth-apt transformation: entailed derivation | see route | see route | see route | see route | see route | Fixed scorer's bounded verdicts about outputs/text | see route | see route | SRC-1, bench/bench_core.py |
EPI-CLM-measurement-scope | none | Truth of vault assertions and semantic answer entailment not checked. |
| EPI-RTE-bench-gate | disposition/acceptance | implemented | EPI-OBJ-evaluation-records | no content change | see route | see route | see route | see route | see route | Acceptance of result consistency for reporting, not truth of answers | see route | see route | SRC-1, bench/validate.py |
EPI-CLM-measurement-scope | none | Rerun equality does not establish that a pipeline explanation is correct. |
| EPI-RTE-diagnostic-checks | check/evidence production | implemented | EPI-OBJ-diagnostic-report | truth-apt transformation: entailed derivation | Graph counts/dates | see route | see route | see route | see route | Parsed graph and age arithmetic under supplied date and filesystem | see route | see route | SRC-1, src/sqlite_graph_memory/vault_stats.py |
none | none | No semantic truth/freshness test. |
| EPI-RTE-diagnostic-checks | check/evidence production | implemented | EPI-OBJ-diagnostic-report | truth-apt transformation: entailed derivation | Environment/import/file-presence predicates | see route | see route | see route | see route | Narrow checked predicates, not verified deployment readiness | see route | see route | SRC-1, src/sqlite_graph_memory/doctor.py |
none | none | A cache directory can exist without valid model bytes. |
| EPI-RTE-diagnostic-checks | disposition/acceptance | implemented | EPI-OBJ-diagnostic-report | non-truth-apt policy/content update: threshold verdict | Health classifications/exit status | see route | see route | see route | see route | No additional truth license beyond checks | see route | see route | SRC-1, src/sqlite_graph_memory/vault_stats.py |
none | none | Thresholds advise repair, do not remove/approve notes. |
| RT-RTE-change-admission | disposition/acceptance | implemented | EPI-OBJ-evaluation-records | non-truth-apt policy/content update: product release choice | see route | see route | see route | see route | see route | Build checks and prescribed regression criterion are distinct | see route | see route | SRC-1, .github/workflows/publish.yml |
none | Documented measurement veto not publisher invariant | Actual human veto and upgrade use uninspected. |
| RT-BAP-temporal-instructions | disposition/acceptance | doctrine only | EPI-OBJ-note-snippets | non-truth-apt policy/content update: close mistaken/expired records | Temporal reliance and human deletion veto | see path | see path | Requested closing, history citation and replacement | Natural-language prescription only | Temporal discipline requested, not factual correctness | see path | see path | SRC-1, BITEMPORAL.md |
RT-CLM-temporal-scope | Host installation/compliance unobserved | No package writer executes the block. |
No lifecycle-integration row is inferred from storage, link expansion or telemetry: within these inspected executable routes, no evidence-consuming acceptance of new ampliative knowledge precedes such integration. This is a bounded route conclusion, not a claim that an external author cannot maintain accepted knowledge.
System-claim versus route comparison
| Claim ID | Claimed operation or warrant | Claim source ID/anchor and evidence layer | Doctrine/design support | Implemented route IDs | Observed-run support | Causal support and design limits | Supported conclusion | Mismatch/unknown |
|---|---|---|---|---|---|---|---|---|
| EPI-CLM-search-layer | see claim | SRC-1, README.md, doctrine/design |
see claim | RT-RTE-index-admission, RT-RTE-mcp-recall | Not assessed | No host-use comparison | Context recall is wired; source truth and agent compliance are not thereby warranted | Read-only describes note recall, not ledger/telemetry writes. |
| EPI-CLM-measurement-scope | see claim | SRC-1, README.md, doctrine/design and reported operation |
see claim | RT-RTE-evaluation-paths, EPI-RTE-bench-score, EPI-RTE-bench-gate | Private numbers attributed only; public result conditions unassessed | Link-derived gold favors walking links; default MCP and benchmark differ | Metrics/checks have operative reporting force, with local reference-dependent license | “What you measure is what runs” holds for shared functions, not all composed policies or answer consumers. |
| RT-CLM-temporal-scope | see claim | SRC-1, BITEMPORAL.md, doctrine/design |
see claim | RT-RTE-recall-options | None supplied for host adherence | Following replacement is not open-record filtering or historical query semantics | Portable temporal discipline is prescribed; limited replacement access is implemented | Closed content can still rank and be delivered as unqualified text. |
| RT-CLM-scoped-return | see claim | SRC-1, src/sqlite_graph_memory/mcp_server.py, implementation |
see claim | RT-RTE-mcp-recall | None supplied | Child success and scoped output do not check statements | Transport contamination protection, not truth validation | No warrant upgrade from error containment. |
Bounded conclusion
The package grants selection authority, not a general epistemic endorsement. Indexed note assertions enter with unknown source warrant; character-preserving clipping and frontmatter removal can degrade contextual warrant. The host sees a bounded excerpt, not all caveats, evidence or temporal metadata. Dense, cross-encoder and graph scores license ranking under their algorithms, not reliance on a proposition's truth (EPI-OBJ-note-snippets, EPI-OBJ-recall-state, RT-BAP-context-delivery). Model parameters are not updated in the supplied inspected inference paths; exact versions and internal factual representations remain unresolved independently of text retention (RT-CMP-e5-model, RT-CMP-rerank-model).
Transcript capture acquires statements and derives a lossy operational ledger. Its later viewer establishes read-back to a person, not automatic answer correction or learning. Selection of a decision-looking line is neither criticism of its content nor validation of the represented action. The parser's invocation records license at most claims about transcript structure, not successful file modification (RT-RTE-turnstate-capture, EPI-OBJ-turns).
The evaluators do produce checkable, bounded propositions: set/rank metrics relative to generated labels; literal citation occurrence and gold-answer matches; numerical consistency of result files; graph/environment predicates. Their warranted premises are the supplied labels, bytes and fixed formal rules, not externally certified world facts. Benchmark answer retention/admission is a score-threshold and nonempty-sentence rule, not acceptance of an ampliative conjecture. Benchmark validation accepts result consistency for reporting; CI status can block continuation under configured workflows, but repository-wide merge enforcement is external. Diagnostics classify and advise through stdout and exit codes, without revising memory (RT-RTE-evaluation-paths, EPI-RTE-answer-abstain, EPI-RTE-bench-score, EPI-RTE-bench-gate, EPI-RTE-diagnostic-checks).
Among the inspected executable routes there is no established produced, accepted ampliative knowledge or post-acceptance lifecycle integration. The contrary candidate—the benchmark's apparent answering—is extractive reshaping plus operational admission, while score arithmetic is entailed derivation in a bounded symbolic domain. Portable instructions can direct external closure and changed reliance, but publication does not establish execution. No independent conclusion about learning, reflection, autonomous theory building or self-improvement follows from persistence or measurement wiring. Product-regression doctrine and release machinery exist; actual evidence-responsive successor adoption and later causal effects remain uninspected (EPI-RTE-extract-answer, RT-BAP-temporal-instructions, RT-RTE-change-admission).
Shared records
Components
none declared in this member
Operative objects
EPI-OBJ-note-snippets
Label: Indexed note snippets
Natural-language note bodies and stored snippets are acquired from caller-selected Markdown files. Paths, dates and titles are symbolic metadata; dense arrays are the separate numerical part. Text and metadata persist in _brain_e5_meta.pkl, vectors in _brain_e5.npy, with a JSON manifest. No runtime object counterpart is declared; closest supplied route is RT-RTE-index-admission, which owns the admission process, not this stored object. SRC-1, src/sqlite_graph_memory/index_notes.py.
body = t[m.end():] if m else t
---
src/sqlite_graph_memory/index_notes.pymeta.append({'path': str(p), 'title': title, 'snippet': c[:SNIPPET_CHARS], 'date': date})
---
src/sqlite_graph_memory/index_notes.py
Clipping is selection of existing characters, not evidence of semantic preservation of an entire assertion. A qualification after character 600 cannot reach recall through that chunk's snippet. The next chunk starts at 1000 under the fixed 1200/200 settings, so delivery is not a guarantee that all omitted material appears in another result. Source-note truth and original evidence lineage are external; path and date establish traceability, not endorsement. Refresh/removal and fast-path hazards are owned by the supplied route.
EPI-OBJ-recall-state
Label: Recall candidates and scores
Dense index/query vectors are distributed-parametric numerical arrays; ranked indices, graph target matches, flags and formatted headings are symbolic. Source-native numerical components are already RT-CMP-e5-model and RT-CMP-rerank-model; this object instead identifies their output/candidate state, not another model. Closest route is RT-RTE-recall-options. Vectors persist in NumPy; per-query lists live in memory and selected text is mirrored to an answer file. SRC-1, src/sqlite_graph_memory/brain_ask.py.
return emb @ qv ---
src/sqlite_graph_memory/brain_ask.pypairs = [(query, meta[i]['title'] + '. ' + meta[i]['snippet']) for i in cand] sc = ce.predict(pairs) if len(pairs) else []
---
src/sqlite_graph_memory/brain_ask.py
No individuated truth-apt world proposition is determinable from the numerical vectors within accessible evidence. Link proximity and pair scores have fixed ranking consumers; neither is an answer oracle. A countercase to temporal exclusion is that the gated branch adds replacements to base_order without removing the superseded seed; see the supplied route's passage. Current-file graph reads and old stored snippets do not form one guaranteed-current assertion set.
EPI-OBJ-turns
Label: TurnState rows
Natural-language ask/summary/decision text and symbolic session, tool, command, file and evidence-pointer fields persist in SQLite turns. No runtime object counterpart; closest process is RT-RTE-turnstate-capture. SRC-1, src/sqlite_graph_memory/turnstate_hook.py, src/sqlite_graph_memory/turnstate_show.py.
summary = "\n".join(summary_parts).strip() for ln in summary.splitlines(): ln = ln.strip(" -*#>").strip() if 8 <= len(ln) <= 180 and DECISION_RX.search(ln): decisions.append(ln) ---
src/sqlite_graph_memory/turnstate_hook.py
Concrete static deduction: an assistant text block Next step: verify cache. is appended to summary_parts, joined unchanged, then passes length and case-insensitive next step matching and becomes a decision string. A Write tool-use input with file_path is added to files without checking any tool-result success. Thus “decision” and “files” are parser classifications of supplied events, not certified commitments or successful writes. This follows implementation, with no fixture or target execution.
if name in FILE_TOOLS: fp = inp.get("file_path") or inp.get("notebook_path") if fp: files.append(fp) ---
src/sqlite_graph_memory/turnstate_hook.py
The viewer selects no evidence/byte-offset columns and prints only a 200-character summary. The original ledger pointer still permits external investigation if the transcript survives; automatic evidential checking is not implemented by this viewer.
EPI-OBJ-evaluation-records
Label: Gold, answers and metric records
Mixed natural-language questions/answers/quotes and symbolic note IDs, reference sets, rankings, metrics and run metadata. Gold and score JSONL, benchmark JSON results and optional SQLite gold_eval retain them; evaluation call state is in memory. No supplied operative-object counterpart; the closest container process is RT-RTE-evaluation-paths. SRC-1, eval/build_gold.py, eval/run_eval.py, bench/run.py, bench/bench_core.py.
elif cls == 'temporal': g, dropped = [n['sup']], 0 else: g, dropped = [n['stem']], 0
---
eval/build_gold.py
Gold labels are constructed expectations, not newly established claims that a linked note really answers the query. Bridge gold is a capped sorted subset of parsed links. Ambiguous stems are excluded by main; this is a countercase to treating every note as an eligible independent label.
stems = all_stems - ambiguous ---
eval/build_gold.pyif os.environ.get('EVAL_MUTANT') == '1': # A mutant run must never enter the history it would later be compared against: a # leftover env var is enough to make every future diff meaningless. print('nothing written: EVAL_MUTANT=1 runs are never persisted') return 0 ---
eval/run_eval.py
This withdrawal concerns intentionally corrupted measurement output, not truth assessment of notes. The record does not adopt any retained result as verified execution or causal evidence.
EPI-OBJ-diagnostic-report
Label: Graph and environment health reports
Symbolic counts, dates, paths, verdict strings and exit statuses, rendered as natural-language lines or JSON to stdout. No persistent repair state is written by the inspected checks. Distinct from RT-RTE-evaluation-paths: targets filesystem/installation health, not retrieval outputs against gold. No supplied object counterpart. SRC-1, src/sqlite_graph_memory/vault_stats.py, src/sqlite_graph_memory/doctor.py.
orphan_pct = round(100.0 * len(orphans) / n, 1) degree = round(len(edges) / n, 2) main, n_comp = largest_component(pages, edges) main_pct = round(100.0 * main / n, 1) ---
src/sqlite_graph_memory/vault_stats.py
The report can entail graph-count propositions under its parsing domain. Age is measured from metadata, not from fact-checking content; unknown dates are not counted as stale. Doctor's file-presence observations have still narrower meaning than its user-facing “all good” language.
Routes
EPI-RTE-extract-answer
Label: Benchmark sentence extraction Part of: RT-RTE-evaluation-paths
This material subroute is distinct from the container's overall retrieval/metric loop: Adapter.ask supplies ranked note IDs and raw notes to best_sentence, which chooses a normalized sentence from the first three notes. Python owns the lexical policy; no human or generative model edits a sentence here. Output is natural-language candidate text with symbolic note identity, in memory until bench/run.py persists the result. SRC-1, bench/adapters/sgm.py, bench/adapters/_extract.py, bench/bench_core.py.
- implementation conclusion status: wired
- operation conclusion status: uninspected
- Immediate return:
(note id, sentence, overlap)to the adapter. - Later read-back: Benchmark result JSON retains returned answer/citation for scoring and reporting; no production recall consumer of this answer is established.
- Delegated visibility: inapplicable — deterministic helper, not a delegated agent.
- Selection predicate: Positive maximum overlap of non-stopword token sets over sentences from first three ranked notes; strict greater-than retains earlier ties.
- Invalidation or expiry: Invocation-local candidate; rerunning can replace the caller's result file, not a knowledge store.
- Activation or effect: Selected text feeds abstention and citation construction; actual run activation uninspected.
- Evidence limits: Normalized substring selection does not evaluate contextual answer correctness or semantic entailment.
for nid in ranked_ids[:depth]: for s in sentences(read_note(nid)): ov = len(qt & tokens(s)) if ov > best[2]: best = (nid, s, ov)
---
bench/adapters/_extract.pytext = WIKI_RX.sub(lambda m: m.group(2) or m.group(1), text or '') return re.sub(r'\s+', ' ', text).strip() ---
bench/bench_core.py
Concrete input traced through inspected code: with only tool-lanternfish ranked, question What language is Lanternfish written in?, and its frozen raw note, frontmatter and heading are removed; punctuation-space splitting produces Lanternfish is written in Elixir. Tokens lanternfish and written overlap the question, whereas the maintenance sentence shares only lanternfish. The first sentence wins, becomes the answer and the same citation quote if the separate score gate permits it. This is non-ampliative selection, not generation of the Elixir claim or proof that the claim is true. No code was executed.
Lanternfish is written in Elixir. Lanternfish is maintained by the [[team-willowherb|Willowherb]] team.
---
bench/vault/tool-lanternfish.md
EPI-RTE-answer-abstain
Label: Benchmark answer admission Part of: RT-RTE-evaluation-paths
Distinct material disposition after EPI-RTE-extract-answer. The adapter proposes the selected sentence; Python decides and vetoes output if the top reranker score is below 0.0 or there is no sentence. The cross-encoder is RT-CMP-rerank-model, not a separate truth oracle. Open benchmark questions use a fixed policy; humans choose the run/configuration, not per-answer admission. No proposal guidance beyond extractor and threshold; no accepted knowledge revision or rollback is involved. SRC-1, bench/adapters/sgm.py.
- implementation conclusion status: wired
- operation conclusion status: uninspected
- Immediate return: Answer/citation object or null answer and empty citations, plus retrieved IDs and top score.
- Later read-back: Caller stores per-question result for later benchmark scoring/validation; not later production knowledge integration.
- Delegated visibility: inapplicable — no reasoning-agent delegation; local model supplies a relevance score.
- Selection predicate: Top reranker score at least ABSTAIN_BELOW and nonempty extracted sentence.
- Invalidation or expiry: Recomputed per question; no stored acceptance state is maintained by the adapter.
- Activation or effect: Enforcing answer suppression at adapter return, not a gate in MCP recall.
- Evidence limits: Top-ranked note's score can admit a sentence from another of the first three notes; there is no sentence-specific truth check.
if top < ABSTAIN_BELOW or not sent: return {'retrieved': ids, 'answer': None, 'citations': [], 'top_score': round(top, 3)} return {'retrieved': ids, 'answer': sent, 'citations': [{'note': nid, 'quote': sent}], 'top_score': round(top, 3)}
---
bench/adapters/sgm.py
Operational admission is permissive/enforcing for this benchmark response only. Its criterion does not consume evidence sufficient to accept a produced ampliative claim; the text was imported/extracted in the first place.
EPI-RTE-bench-score
Label: Benchmark reference checking Part of: RT-RTE-evaluation-paths
Distinct material checker: score_one and aggregate functions evaluate per-question outputs against caller-supplied question answers/support IDs and normalized vault text. bench/run.py calls score, displays the metrics and persists them. Fixed Python owns comparison; question records provide the answer oracle. This is bounded evaluation, not per-note admission or a human approval workflow. SRC-1, bench/bench_core.py, bench/run.py.
- implementation conclusion status: wired
- operation conclusion status: uninspected
- Immediate return: Recall, lexical answer hit, citation occurrence/support/answer-hit and abstention verdicts aggregated overall/by class.
- Later read-back: Metrics and per-question results persist in benchmark JSON;
check_resultrederives them for validation. - Delegated visibility: inapplicable — deterministic scorer receives returned outputs, gold and normalized vault.
- Selection predicate: Fixed support sets, first k deduplicated retrieved IDs, bounded deduplicated citations; lexical word-boundary answer matching.
- Invalidation or expiry: Dependent on current questions, vault and outputs; no general validity beyond those inputs.
- Activation or effect: Produces displayed/persisted metrics used by the separate result gate; no automatic production-policy change.
- Evidence limits: Formal consistency/lexical matches do not certify source truth, answer entailment, completeness or real execution.
ok = bool(quote) and nid in vault and quote in vault[nid] faithful.append(ok) if (ok and answerable and len(quote) >= MIN_QUOTE and nid in support and contains_answer(quote, q['answers'])): verbatim_hit = True
---
bench/bench_core.py
A countercase to broad “faithful” semantics is a literal quote used with a wrong or contradictory answer: citation_faithful tests quote occurrence irrespective of that answer. answer_ok separately tests presence of a gold string, not negation or entailment. License is therefore exactly the implemented predicates. No causal result is inferred from this arithmetic.
EPI-RTE-bench-gate
Label: Benchmark result acceptance
Distinct from RT-RTE-evaluation-paths's evaluation execution: this route admits/rejects a result claim for reporting, consuming scorer results, hashes and result structure, and optionally independent rerun outputs. It overlaps the checker EPI-RTE-bench-score but owns disposition, not metric production. Result submitter proposes; Python checks can veto; configured CI consumes nonzero status. Human merge powers/branch protections are external. Trigger is explicit CLI/CI invocation, not an adaptive recall loop. SRC-1, bench/bench_core.py, bench/validate.py, .github/workflows/bench.yml.
- implementation conclusion status: wired
- operation conclusion status: uninspected
- Immediate return: Printed validation/reproduction findings and success/failure exit status.
- Later read-back: Committed result files remain; configured reproduction preserves rerun artifacts for diffing. Gate itself does not rewrite results or a production model.
- Delegated visibility: Reproduction subprocess runs the benchmark and returns a new result file; no delegated agent evaluation.
- Selection predicate: Shape/coverage/hash consistency, recomputed metric equality, generated vault/leaderboard consistency in ordinary full validation; optional rerun metric tolerance plus zero changed per-question keys.
- Invalidation or expiry: Changed vault/question hashes or mismatched results trigger rejection; correction/rerun is caller-owned, no automatic rollback of deployed code.
- Activation or effect: Nonzero command status fails its CI step;
reproducejob depends onvalidate. Merge enforcement depends on external repository settings. - Evidence limits: No gate or CI execution observed; self-awarded provenance is limited by configured list membership, and content consistency alone cannot authenticate execution.
need(res.get('vault_sha256') == vault_sha, 'vault_sha256 does not match bench/vault (stale result or edited vault)') need(res.get('questions_sha256') == questions_sha, 'questions_sha256 does not match bench/questions.jsonl') ---
bench/bench_core.pyif worst > tol or changed > MAX_CHANGED: print('FAIL the committed outputs for %s do not reproduce' % system); return 1 print('ok %s reproduces' % system) return 0 ---
bench/validate.pyWhat it cannot catch: per-question outputs that were themselves made up. That is why a result says whether CI reproduced it (
reproduce.in_ci), and the board shows it. ---bench/validate.pyreproduce: runs-on: ubuntu-latest needs: validate ---
.github/workflows/bench.yml
Acceptance criterion and use are bounded: consistent report relative to the frozen benchmark inputs, or reproduction agreement in the selected rerun mode. Neither accepts an external factual theory. A passing rerun supports repeatability of compared outputs under its environment, not component-level causal benefit or general transfer.
EPI-RTE-diagnostic-checks
Label: Health checks and verdict consumers
No supplied counterpart after comparison: runtime explicitly leaves sgm-stats/sgm-doctor detail unassessed. These operator-invoked deterministic routes assess graph structure/date metadata and machine import/cache/index presence, not retrieval against gold. Python owns checks and threshold verdicts; callers choose thresholds/date and whether to act on advice. There is no per-content human approval, reference answer oracle, successor selection or automatic repair. Check and disposition functions are separated in the ledger. SRC-1, src/sqlite_graph_memory/vault_stats.py, src/sqlite_graph_memory/doctor.py.
- implementation conclusion status: wired
- operation conclusion status: uninspected
- Immediate return: Stats text/JSON and exit 0/1/2; doctor lines/fixes and exit 1 for failures, otherwise 0.
- Later read-back: inapplicable — these entry points do not persist their reports; external cron/log capture is outside inspected code.
- Delegated visibility: inapplicable — no agent delegation; doctor imports installed packages as checks.
- Selection predicate: Shared note enumeration/link parsing, resolved edges, concept classification and metadata date cutoff; environment/module availability, cache-directory existence, pairwise index-file presence and version predicates.
- Invalidation or expiry: Reports are snapshots; rerun after filesystem/environment/date changes. No note content is invalidated.
- Activation or effect: Code emits warning/failure and exit status; these can advise a person or be consumed by a watchdog. Actual repairs or watchdog behavior uninspected.
- Evidence limits: Threshold verdicts do not warrant truth of note assertions, model integrity or successful end-to-end recall; doctor does not deserialize/check index contents.
stale = sorted(s for s, d in dated if d < cutoff) if concepts: stale_pct = round(100.0 * len(stale) / len(concepts), 1) stale_v = 'warn' if stale_pct > th['stale_warn'] else 'ok'
---
src/sqlite_graph_memory/vault_stats.pyworst = max(RANK[m['verdict']] for m in metrics) return {'notes_dir': str(notes_dir), 'pages': n, 'metrics': metrics, 'verdict': ['ok', 'warn', 'fail'][worst], 'exit_code': worst}
---
src/sqlite_graph_memory/vault_stats.pyif not any((d / folder).exists() for d in _hf_cache_dirs()): missing.append(name) ---
src/sqlite_graph_memory/doctor.pytext, fails = render(run_checks()) print(text) return 1 if fails else 0 ---
src/sqlite_graph_memory/doctor.py
The contrasting cases bound authority: an undated concept is excluded from the stale numerator but included in its denominator; an existing model-cache folder passes its presence check without proving complete weights; two existing index files pass presence checking without proving compatible rows. These are limits on target/domain, not absent evaluators.
Claims
EPI-CLM-search-layer
Label: Read-only associative recall
Source-native claim is context retrieval over notes, rather than independent answer production. Closest supplied claim RT-CLM-scoped-return concerns transport isolation; this is the distinct functional/warrant boundary. SRC-1, README.md, doctrine/design; implemented comparison uses RT-RTE-mcp-recall.
It fits Andrej Karpathy's LLM Wiki pattern (April 2026), where an agent maintains an interlinked markdown wiki and a search tool over the pages is the first helper he suggests adding: sgm is that search layer, read-only, and it follows your
[[wikilinks]]as a graph. Independent project, not affiliated with or endorsed by him.---
README.md
Read-only concerns source notes on recall, not all package side effects. The index and ledger retain content and telemetry. No claim that graph association validates note truth is established by this search-layer statement.
EPI-CLM-measurement-scope
Label: Measurement and reproducible evidence
Claims concern what evaluation can establish, what shared functions reproduce, and private reported performance. Distinct from RT-CLM-temporal-scope, whose target is temporal reliance. SRC-1, README.md, doctrine/design and reported operation. No new execution evidence is asserted here.
run_eval.pyimportsbrain_ask.pyand calls the same functions the agent calls —load_index,retrieve_candidates,expand_1hop,rerank_candidates— so what you measure is what runs. An eval that re-implements the pipeline measures the re-implementation. ---README.mdWikilink gold is weak supervision: a linked neighbour is not always the note that answers the question.
bridgeis built from links, so any method that walks links is flattered there — it measures "can the pipeline follow a connection", not "is this the right connection". ---README.mdBe clear about what that reproduces: the bridge numbers above came from the maintainer's home vault and home pipeline on 2026-09-16, and they have not been re-run from the published package. ---
README.md
Shared functions support algorithm reuse, not equivalent composition. A countercase is the supplied runtime's entity-policy divergence and the benchmark's mandatory reranking/extractive answering. Private figures are attributed operation, not authenticated observed/causal evidence for this package. The README itself explicitly restricts the license of its gold and private figures; the report retains those restrictions rather than treating evaluation as universally invalid.
Evidenced absences
none declared in this member
Behavioral-authority paths
none declared in this member