Napkin report verification

Type: agentic-system-analyses/types/agentic-system-verification.md

Verification

The pinned reports support a bounded account of Napkin as a file-based memory system with callable retrieval and mutations, external-agent curation instructions, and distinct benchmark ingestion/answer consumers. No material correction is required. This accepts the source account, not successful distillation, safe scheduled operation, reported accuracy or causal benefit.

I read all five reviewed members, their contracts, the boundary and the supplied check. The four correction-answer inputs are empty; the structural report-check is none. There is no structural failure to route and no correction argument to accept by default. I independently checked the frozen checkout's HEAD and clean tracked status, inspected its tracked tree, and traced consequential behavior in SRC-1. It remains at the boundary's registered revision. No target, test, example, benchmark or model was executed. The Source register correctly distinguishes source implementation/design from unavailable traces, datasets, external extension and provider implementations.

Record support and evidence strength

Runtime and authority. RT-CMP-napkin, RT-OBJ-vault-info, RT-RTE-vault-retrieval and RT-RTE-caller-mutation match the package exports, SDK constructor/delegation, discovery, CLI wrappers and core effects. SRC-1, package.json, src/sdk.ts, src/main.ts, src/utils/vault.ts, src/commands/search.ts, src/commands/crud.ts, src/core/search.ts, src/core/crud.ts and src/utils/files.ts support the described principal, next-step ownership and return channels. Construction can create a bare vault before query validation. Create's guard is branch-scoped and non-atomic; direct overwrite and permanent deletion remain exposed. Path joining supplies no containment assertion. Trash is excluded from walkers but remains readable by an exact path. These findings do not imply an executed exploit, transaction, host permission policy or semantic approval.

RT-RTE-machinery-change matches SRC-1, src/utils/config.ts, src/core/config.ts, src/core/init.ts, src/templates/index.ts, src/commands/update.ts and .github/workflows/release.yml: copied/frozen injected settings, refused configSet on injection, caller registry replacement, missing-file scaffold writes and test-gated publication are distinct admission mechanisms. Mutable npm latest is not the reviewed revision. The legacy forwarders in legacy/napkin-ai/cli.js and legacy/napkin-ai/index.js justify delegation only, not behavioral equivalence. RT-CMP-pi-model correctly separates the requested model identifier from inaccessible weights, endpoint resolution and parameter updates.

Retained parts and selectors. MEM-OBJ-markdown-notes, MEM-OBJ-napkin and MEM-OBJ-note-templates preserve the different consumers of prose, context, symbolic metadata and template bytes. SRC-1, src/core/crud.ts, src/core/overview.ts, src/core/daily.ts, src/core/templates.ts, src/templates/coding.ts and src/utils/files.ts support retention and later reads without guaranteed rationale, provenance or host injection. Template creation/daily copying differs from variable-resolving insertion.

MEM-OBJ-search-cache, MEM-OBJ-overview-cache and RT-CLM-keyword-probes match SRC-1, src/core/search.ts, src/core/overview.ts, src/utils/fingerprint.ts, src/utils/search-cache.ts, src/utils/overview-cache.ts and src/commands/overview.ts. Search retains a dependency index and metadata, then rereads bodies for snippets; overview retains its display/context result. The path/mtime keys do not authenticate content or endorsement. Folder fingerprints omit outside-folder backlink changes. I checked both failed-probe fallback and roster completion, not just the successful-probe branch. The reports correctly withhold a universal search-validation guarantee and a shared model-token budget.

MEM-OBJ-base, MEM-OBJ-canvas, MEM-OBJ-bookmarks, MEM-RTE-auxiliary-edits and MEM-RTE-structured-access have supported persistence/consumer distinctions in SRC-1, src/core/bases.ts, src/core/canvas.ts, src/core/bookmarks.ts, src/core/daily.ts, src/core/properties.ts, src/core/tasks.ts, src/core/templates.ts and src/utils/frontmatter.ts. Base queries close the transient database; baseCreate bypasses the ordinary create guard; canvas references do not themselves load note/URL content; bookmark parse failure returns an empty list. Partly assessed expression semantics are not presented as a complete query account.

Agent instructions and workflow. RT-RTE-distill-session, RT-RTE-tend-vault and RT-BAP-skill-guidance are supported policies, not core enforcement or observed agent behavior. I read both complete skills in SRC-1, skills/distill/SKILL.md and skills/tend/SKILL.md. KEEP/SKIP, bounded duplicate search, contradiction treatment, inference marking, conservative upkeep, human schema choice and requested reporting are correctly assigned to their consumers. MEM-RTE-background-distill and EPI-CLM-background-distill accurately retain the separate-model doctrine in docs/distill.md, rather than importing an unavailable scheduler into the shipped implementation.

The workflow account is one connected skill procedure, not a composite assembled from benchmark capabilities. WF-RTE-distill-verification, EPI-OBJ-link-index and EPI-RTE-link-check match SRC-1, src/core/links.ts, src/utils/markdown.ts, src/utils/files.ts and src/commands/links.ts. The diagnostic follows persistence, resolves recognized wikilinks with loose ambiguity handling, strips fragments and does not block or roll back writes. Default output omits referring-file provenance. Caller repair and final reporting are requested, not demonstrated. I also traced the skill's create example through CLI registration to createFile: both the folder/file-path mismatch and unpopulated template-copy limit are real and already disclosed.

Epistemic transformations. EPI-OBJ-vault-prose, EPI-OBJ-base-values, EPI-RTE-property-edit, EPI-RTE-template-insert and EPI-RTE-base-summary retain appropriate scopes. SRC-1, src/core/properties.ts, src/utils/frontmatter.ts, src/core/templates.ts, src/core/bases.ts and the query/summary ranges of src/utils/bases.ts support import, substitution, retention and conditional numeric computation. The finite 2-plus-3 witness establishes the narrow arithmetic deduction, not source-premise truth or arbitrary formula fidelity. Status metadata is not evidence-consuming acceptance. The ledger separates checking from disposition and retention from accepted lifecycle integration. Its indeterminate semantic relation for model-produced notes avoids equating unproved entailment with conjecture. EPI-CLM-distill-knowledge and EPI-CLM-tend-safety are assessed against actual instruction force, not treated as guarantees.

Benchmarks. RT-RTE-benchmark-answer, RT-BAP-benchmark-evidence, MEM-RTE-benchmark-variants, MEM-OBJ-session-summary, MEM-OBJ-evaluation-results, EPI-OBJ-benchmark-notes and EPI-OBJ-answer-score match the distinct producer/consumer paths in SRC-1, bench/longmemeval-eval.ts, bench/longmemeval-prompt.md, bench/locomo-eval.ts and bench/hotpotqa-eval.ts. I checked ingestion, pi arguments, text/tool parsing, scoring and cleanup/report consumers. LongMemEval writes raw rounds with date-derived mtimes and tracks first-round proxies; LoCoMo imports supplied summaries and reuses a sample vault; HotpotQA writes paragraphs and heuristic links. All disable skills. Prompt restrictions do not demonstrate sandbox enforcement. Retained scores are reporting inputs, not inspected next-question knowledge updates.

EPI-RTE-dataset-import, EPI-RTE-answer-extract, EPI-RTE-longmem-judge, EPI-RTE-token-score and EPI-RTE-score-report correctly expose evaluation's limited license. I independently traced the qualification-removing extraction, forward/reverse substring full-credit counterexamples, temporal tolerance, continuous fallback and word-order-insensitive token calculation. These are implementation deductions, not observed misgrading. Completed non-null results feed aggregates; raw failure cases do not become accuracy zeros. EPI-CLM-benchmark-results matches SRC-1, README.md; its reported headline and historical comparison constants are not reproduced results or causal contrasts.

I checked every declared record's referent, evidence layer and consequential status, the lens annotations and the three explicit Part of: relations. The two LongMemEval check/report parts belong to the benchmark container; workflow post-write verification belongs to distill. The reconciliation correctly retains overlapping prose/results containers and narrower operations without supersession. No amendment is asserted, no evidenced-absence record is declared, and no unsupported absence status is used to close an uninspected route. The background producer difference, trash exact-read exception, auxiliary write guard difference, scorer defect and create-example mismatch are qualifications, not unresolved disagreements between analysts. No conflict requires a reconciliation correction.

Omission questions were selected from the tracked tree and producer/consumer dependencies: could an alternate writer turn skill admission into a universal gate; could a shipped benchmark preserve derived content or reuse state; could cache/overview branches invalidate freshness and validation guarantees; could an auxiliary evaluator or graph consumer overturn the bounded epistemic conclusion? Direct reads of the auxiliary writers, all three benchmark driver paths, cache consumers and overview exceptions dispose the first three questions. A scoped SRC-1 search across src/ for writes, configuration and distillation consumers found no additional core scheduler to attribute to the packaged skills. This search supports the narrow inspected-chain conclusion, not system-wide absence.

For the last question I inspected base query/summary dispatch and graph browser delivery in SRC-1, src/utils/bases.ts and src/commands/graph.ts, in addition to the supplied coverage. Formula and renderer internals remain incompletely assessed, as the reports already state. Their potential additional derivation/display functions do not negate the positive retained-state account or the explicitly non-complete epistemic negative. Further optional expansion stopped because each selected challenge has a supported disposition and no emitted central claim depends on exhaustive auxiliary semantics. The remaining incomplete comparisons are retained below, not recast as absence. External injection, run compliance, actual provenance and benefit remain faithfully scoped uncertainties beside the affected findings; they are not repaired by this verification.

Blockers

none

Limits

  • The distill create example has a source-level delivery mismatch already disclosed in the workflow account and its mutation annotation, concerning RT-RTE-caller-mutation and MEM-OBJ-note-templates. SRC-1, skills/distill/SKILL.md, src/main.ts and src/core/crud.ts treat the example's folder-valued path as a complete filename and copy template bytes without populating distilled content. Withhold successful end-to-end new-note delivery from literal execution of that example. The source-supported selection, merge, mutation, diagnostic and reporting procedure remains characterizable; the reports do not assert successful execution.
  • The overview exposure scorer cannot serve as verified executable validation at this pin: memory integration issues retain the string-argument/VaultInfo signature mismatch in SRC-1, bench/overview-exposure.ts and src/core/search.ts. Withhold scorer-based keyword-quality or exposure conclusions. RT-CLM-keyword-probes remains supported by the separate overview path and its explicit fallback exceptions.
  • Auxiliary semantic inventories remain partial: MEM-RTE-structured-access, EPI-RTE-base-summary, the memory coverage and epistemic coverage leave formula/filter, tokenizer and graph-rendering details incompletely assessed. SRC-1, src/utils/bases.ts, src/utils/formula.ts, src/core/overview.ts and src/commands/graph.ts are available, not access gaps. Withhold exhaustive derivation/evaluator, UI-consumption and selector-quality comparisons. The narrower numeric witness, explicit storage consumers, request-driven retrieval and instruction-versus-enforcement distinctions remain valid.