xMemory report verification

Type: agentic-system-analyses/types/agentic-system-verification.md

Verification

The reports support a bounded implementation account, with one local evaluation-inventory omission retained below. I read all four analyst members, reconciliation, their types, the record contract and the supplied check. Correction-answer inputs were empty; the structural report-check says none. There are no structural failures to route. I independently inspected source rather than accepting the analysts' validation claims. SRC-1 HEAD matches the boundary, and its checkout is clean. I did not execute the target, its examples or providers. Implementation, requested model behavior, observed operation and benefit remain separate.

Runtime and workflow

  • RT-RTE-conversation-admission and EPI-RTE-episode-normalization: traced dataset formatting, buffer branching, clear-before-generation, episode creation, fallback, save/index order and event dispatch. SRC-1, evaluation/locomo/add.py, src/core/memory_system.py, src/core/boundary_detector.py and src/generation/episode_generator.py support the account. The smart-boundary branch bypasses the maximum-size elif; confidence does not supply a second gate. Normalization's non-ampliative finding is supported for its explicitly stated whitespace-only case, not a fidelity guarantee for generated episodes or placeholder substitution.
  • RT-RTE-semantic-admission: followed event scheduling, prediction-correction/per-episode dispatch, saving, callback accumulation, polling and timeout cleanup in SRC-1, src/core/memory_system.py and src/generation/semantic_generator.py. A zero-active-task result is not successful hierarchy completion. Driver maintenance follows a snapshot and ignores the wait result. The wired/uninspected statuses and non-atomicity qualifications are supported.
  • RT-RTE-hierarchy-admission: checked driver ordering and the new-manager/new-batch handoff through theme maintenance and graph export in SRC-1, evaluation/locomo/add.py, src/core/memory_system.py and src/core/memory_hierarchy.py. Numerical structural admission and generated summaries are distinct. Source wiring does not establish complete provenance or faithful abstraction.
  • RT-RTE-baseline-search: traced facade/async dispatch, zero-count branches, Chroma results, NOR-LIFT fallback and direct lightweight loading in SRC-1, src/api/facade.py, src/core/memory_system.py, src/search/chroma_search.py, src/search/unified_search.py and src/core/lightweight_search_system.py. Memory's qualifications correctly prevent interpreting the dispatch name as an implemented aggregation algorithm or baseline delivery as unconditional original-message hydration.
  • RT-RTE-adaptive-reader, RT-CLM-retrieval-purpose and RT-BAP-memory-reader: independently traced hierarchy reload, representatives, candidate construction, fixed-baseline scoring, thresholds, early stops, fallback, hydration and final prompt in SRC-1, evaluation/locomo/xMemory_search_framework.py; compared the claim with README.md. The mean generated-token NLL is not a correctness or final-answer utility oracle. Expansion is decided without testing originals, and the final answer excludes themes. Positive-only block caps are not input-token caps. The reports preserve these differences rather than endorsing the README's stronger “only when” claim.
  • RT-CMP-llm-client, RT-CMP-embedding-client, RT-RTE-provider-admission and RT-ABS-action-dispatch: checked default exports, factories, injected implementations, model loading, inference and embedding requests in SRC-1, src/utils/__init__.py, src/utils/llm_client_openllm.py, src/utils/embedding_client.py, src/services/providers.py and src/api/facade.py. Repeated the dispatcher search across src/, evaluation/ and xMemory/ for execution, tool, approval and loader terms. Text-returning model calls support the bounded dispatcher absence; trusted remote loading and arbitrary host injection prevent a sandbox conclusion. Named models/endpoints do not pin weights, and inference does not establish parameter training.

The workflow account and annotations add supported failure and delivery distinctions without declaring competing records. In SRC-1, evaluation/locomo/add.py, a returned add error is not an exception-driven retry, and exhausted None flush results stop maintenance. In evaluation/locomo/xMemory_search_framework.py, generation failure can become an empty saved response, whereas an escaping worker exception skips the normal save call. Sequential questions, conversation-level workers, cleanup and separate sidecar writes do not establish agent delegation, answer acceptance, replay safety or atomic delivery.

Retained parts and epistemic functions

I checked each object declaration against its producer and actual consumer, including the duplicate identities discussed below:

  • MEM-OBJ-original-messages, MEM-OBJ-episode-content, MEM-OBJ-semantic-content and MEM-OBJ-memory-metadata: SRC-1, src/models/episode.py, src/models/semantic.py, both generation implementations and the reader preserve the distinction between originals, derived prose and access/provenance fields. A source episode ID permits navigation but does not supply admission reasons or authenticate conversation truth. Fixed confidence and append/skip-existing-ID persistence do not establish repair.
  • MEM-OBJ-theme-summary, MEM-OBJ-theme-topology, MEM-OBJ-retrieval-vectors and MEM-OBJ-semantic-knn: checked fresh empty caches, batch-local summary refresh, split/merge/coalescence, local vector persistence, new-entry adjacency updates and reader consumption in SRC-1, src/core/memory_hierarchy.py, src/core/memory_system.py and evaluation/locomo/xMemory_search_framework.py. The old-member omission and potentially stale neighborhoods are real source-level cases, not measured occurrences. Text, symbolic filtering and dense numerical selection have different force.
  • MEM-OBJ-memory-graph and MEM-OBJ-original-index: SRC-1, src/core/memory_hierarchy.py, src/search/original_message_search.py and src/search/episode_original_message_search.py support a retained diagnostic export and separate callable original-text selectors. The export's batch maps limit edge creation; it is not the adaptive reader's substrate. A caller search found no shipped QA use of the original-message helpers. That supports the stated capability/consumer distinction, not absence of possible external consumers.
  • MEM-RTE-memory-withdrawal and EPI-RTE-owner-deletion: checked owner deletion, store behavior, explicit cache no-op and independent reader cache in SRC-1, src/core/memory_system.py, src/storage/semantic_storage.py and evaluation/locomo/xMemory_search_framework.py. The reports correctly withhold all-layer revocation and distinguish administrative deletion from evidential refutation.

The remaining epistemic objects have distinct referents: EPI-OBJ-predicted-episode is transient prediction prose; EPI-OBJ-entropy-score is current-query numerical control; EPI-OBJ-qa-result is generated response content; and EPI-OBJ-reference-answer is an imported offline oracle. Their horizons and consumers agree with SRC-1, src/generation/prediction_correction_engine.py, evaluation/locomo/xMemory_search_framework.py and evaluation/locomo/evals.py.

For EPI-RTE-comparison-extraction, EPI-RTE-duplicate-check, EPI-RTE-semantic-disposition and EPI-RTE-semantic-retention, the source separately supports candidate generation, cosine redundancy checking, insertion permission and persistence. SRC-1, src/generation/prediction_correction_engine.py, src/generation/prompts.py and src/core/memory_system.py do not establish independent truth acceptance merely through this progression. The comparison consumes originals, but returns statement strings without a retained challenged-claim/blame decision. Indeterminate semantic relations are appropriately bounded; no general claim that model generation must be ampliative is emitted.

RT-RTE-offline-evaluation and its part EPI-RTE-lexical-evaluation match the README scoring route: category 5 is skipped, F1 uses token sets, the model-judge call is commented, and output is consumed for reporting. I checked SRC-1, evaluation/locomo/evals.py, evaluation/locomo/metrics/utils.py and evaluation/locomo/generate_scores.py. This supports the narrow lexical-license account and no automatic score-to-memory revision on that route. The alternate judge omission below prevents extending that inventory to every shipped evaluator.

EPI-CLM-faithful-hierarchy accurately attributes SRC-1, README.md. Retaining originals and organizing derived material do not independently verify summary fidelity. Its comparison with the hierarchy route and the bounded theory/learning conclusions do not claim successful criticism, improved capacity or causal benefit from implementation alone.

Relations, integration and omission stopping

I checked all six declared Part of: relations: normalization is within conversation admission; comparison, duplicate checking, disposition and retention are within semantic admission; lexical evaluation is within offline evaluation. Each is a contained function, not a candidate substitute for its parent. The parents remain valid containers.

The four amendments in reconciliation identify the same originals, semantic content, theme summary and owner-deletion route across members. They preserve the epistemic qualifications rather than transferring stronger evidence status. Its remaining overlaps are also sound: EPI-OBJ-episode-content isolates content within memory's title/content container; EPI-OBJ-selection-state isolates membership within topology; EPI-OBJ-embedding-vectors overlaps memory's broader vector/centroid inventory. No supersession of these valid containers is required. I checked all six memory integration issues and workflow's failure qualifications against reconciliation's dispositions. No unresolved cross-report contradiction or unsupported status escalation remains. Empty correction answers provide no additional repair claims to assess.

Selected omission questions came from the tracked tree and producer/consumer dependencies: whether facade-only inspection missed shipped maintenance/answer callers; whether caches, alternate extraction or original-message helpers change trace lineage or read-back; whether deletion implies universal withdrawal; and whether additional evaluators change the claim that scoring is lexical and offline. Direct source tracing above settles the first three without changing the emitted bounded findings. I additionally inspected SRC-1, src/utils/performance.py: cached-call keys incorporate supplied arguments, so the coarse function-name prefixes alone do not demonstrate content-blind episode reuse. The evaluator search found the standalone judge described below.

Further optional expansion stopped because these challenges now have supported dispositions or a retained local limit. Every uninspected backend, objective term or tokenizer variant is not needed to support the narrower claims actually emitted. No exhaustive ranking-equivalence, all-evaluator inventory, global consistency, deployed recovery or successful-execution conclusion is accepted. Existing scoped uncertainty about these matters is not evidence of absent behavior.

Blockers

none

Limits

  • The evaluator inventory is incomplete: RT-RTE-offline-evaluation, EPI-RTE-lexical-evaluation and the workflow account faithfully describe evals.py, but do not characterize the separate executable judge in SRC-1, evaluation/locomo/metrics/llm_judge.py. It reads saved question/reference/response records, calls gpt-4o-mini, maps its label to a number, writes per-item labels and prints category accuracy. Its main loop calls label = evaluate_llm_judge(question, gold_answer, generated_answer); the function's label consumer is shown below. This is another offline assessment/report route, not evidence of a memory-admission or automatic revision loop. The existing route-local wording remains supported, so this is not a blocker. Withhold complete comparisons of evaluator coverage, model-judged correctness criteria and evaluator-model provenance; do not generalize the commented call in evals.py to absence of a shipped model judge.

label = json.loads(response.choices[0].message.content)["label"] return 1 if label == "CORRECT" else 0 --- evaluation/locomo/metrics/llm_judge.py