Hindsight synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From submitted experience to retained state
Hindsight is a memory service with its own model-driven retrieval-and-answer loop and shipped callers that retain experience or deliver memories to external agents. This account is code-grounded in SRC-1, https://github.com/vectorize-io/hindsight, at commit 610c135b6f2d0d8b972da94146ae635943f36271, with an applicability cutoff of 2026-10-14. It covers a complete repository artifact with partial loop inspection, not a whole-system audit. Implementation and in-repository design claims support the account; no supplied execution trace or causal experiment establishes successful operation. External host orchestration, hosted Cloud internals and model-provider internals are outside the inspected loop. See the boundary and evidence and Source register.
The strongest supported experience-to-future-action contribution is a wired, recurring transcript-to-memory-to-context path. Claude Code Stop hooks select conversation suffixes or turn windows, optionally including tool-call material, and submit them for retention. Later UserPromptSubmit hooks form recall queries, filter and format results, and emit additional context to the host. Durable extracted memory, rather than just a raw log, can therefore shape later model inputs. This establishes automatic trace-fed artifact production and a later delivery channel, not observed host reliance or improved capacity. Local retention progress can advance after client return, or on an empty-content branch; it is not proof of completed server ingestion. See RT-RTE-claude-hooks and RT-BAP-hook-context.
Retention normally transforms submitted text into extracted facts before embedding and persistence. Optional original document/chunk storage is distinct from fact text; “raw” facts are pre-consolidation assertions, not necessarily verbatim transcripts. Extraction can retain a supplied reason in prose without preserving the complete reasoning episode. Optional operation validators can replace or reject ingress. Defense screening operates only when the extension and policy are enabled; its allow/redact/block decision concerns configured sensitive-data policy, not factual truth. Synchronous retention returns produced IDs, whereas queued retention initially returns an operation acknowledgement. Worker retry and terminal-state handling provide local recovery, not an exactly-once whole-pipeline guarantee. See RT-RTE-retain-work, MEM-OBJ-fact-text, MEM-OBJ-document-text and EPI-RTE-defense-screen.
Revising reusable memory
Admitted facts can feed observation consolidation. A configured model proposes creates, updates and deletes over new facts and related recalled observations. Code restricts targets and sources and commits or aborts prepared actions. This is implemented source-conditioned revision; source IDs and counts identify derivation, not independent corroboration. Model instructions to consolidate without deductions do not establish faithful preservation of arbitrary outputs. See MEM-RTE-observation-consolidation and the epistemic assessment.
Persistent mental models are standing documents; knowledge-tree pages reference those backing documents. Creation can store caller-supplied Markdown. Refresh can later regenerate the document or apply localized edits from scoped evidence. Knowledge pages default to observation-only, delta, automatic refresh unless overridden. Named failures preserve the prior document and watermark rather than installing a delta-window answer as a whole-page replacement. Stored grounding helps detect withdrawn sources; optional histories and traces support audit, but ordinary page reads do not send the entire historical rationale to the answering model. These mechanisms establish durable revision and procedural admission, not truth acceptance. See MEM-RTE-page-maintenance, EPI-RTE-page-refresh and MEM-OBJ-refresh-response.
Withdrawal changes the dependency chain. On the inspected PostgreSQL path, outgoing raw facts cause dependent observations and their history to be removed, while surviving co-sources are reset for reconsolidation. Archive-based invalidation preserves restoration state and an optional reason, unlike permanent deletion. Page-refresh nudges after curation are best effort, cover automatic pages and can defer behind pending consolidation; manual pages require their owner's action. A successful curation response therefore does not prove that every page or already delivered context has stopped repeating the fact. See MEM-RTE-memory-curation, EPI-RTE-source-withdrawal and MEM-RTE-page-maintenance.
Selecting evidence and delivering an answer
Recall selects retained facts and observations using query, bank and filters, then ranking and budgets. Fact text, chunks and source facts have separate delivery controls. A source ID can survive when its readable text does not. Page search can expose one fitting best hit whole and other hits as snippets; explicit reads reload selected documents. Positive-budget first-item floors can exceed the requested fact or page budget. Retained lineage, complete readable justification and a strict whole-context ceiling are therefore different claims. See MEM-RTE-recall-selection and MEM-RTE-page-delivery.
The representative multi-step workflow is HTTP reflection: a caller supplies bank, query and controls; the engine resolves configuration, identity and directives; code initiates retrieval; the model proposes subsequent calls or completion; Hindsight dispatches enabled callbacks and returns their results to the next model call. The effect boundary is retrieval and provider calls, not arbitrary execution of the host's task. With the Claude Code provider, SDK handlers collect proposals and Hindsight still executes retrieval. This is not delegation to independent autonomous task agents. See RT-RTE-reflect-request and RT-RTE-provider-round.
Ordinary completion without gathered evidence IDs can be returned for more retrieval. Document-mode completion adds structural validation and bounded corrective feedback. Neither checks whether answer propositions follow from the evidence. Length rewriting and ID filtering can change the proposed answer; repeated document-rewrite rejection preserves the original, potentially over budget. HTTP delivery means returning postprocessed text and optional support, usage, traces or structured-output errors, not client acceptance or automatic answer retention. See WF-CLM-completion-checks.
Iteration or context exhaustion changes the workflow. Older model-visible tool bodies can be blanked while a separate current-request history remains available. Terminal synthesis can partition that history, compress slices in separate model calls and reduce generated claim sections. This controls working context without establishing lossless compression or a durable cross-request reload chain. It also prevents treating the initial retrieval sequence as a universal grounding invariant. See RT-RTE-context-synthesis.
Authority remains consumer-specific. Retrieved prose supplies interpretive evidence; active directives supply standing imperative text, not demonstrated semantic enforcement. Bank overrides select runtime settings through fixed rules. Authentication is configurable: the default tenant implementation is no-auth, and trusted internal or MCP-marked contexts bypass revalidation. Startup extensions admit executable Python into the process; subclass validation is not a safety sandbox. These qualifications matter to any scenario assuming enforced policy or isolation. See MEM-BAP-reflect-content, MEM-BAP-directive-prompt, RT-RTE-config-update, RT-CLM-tenant-check and RT-RTE-extension-load.
What the loop establishes
For recurring personalized context and reusable standing answers, the inspected architecture supplies automatic admission, durable revision and selective later delivery. For a scenario requiring independently accepted world claims, immediate universal retraction or hard output bounds, those mechanisms do not establish the required guarantees. Public descriptions of evidence-backed beliefs, settled knowledge and smarter agents remain stronger than the conclusions supported by storage and source-conditioned revision alone. See EPI-CLM-memory-beliefs and MEM-CLM-source-hierarchy.
There is a separate, consequential correctness evaluator. Knowledge-page convergence tests grade final pages against question-specific criteria and forbidden claims, record verdicts and fail assertions. Initial failing model verdicts can be reasked and majority-overruled; this is not calibrated confidence. The criterion-relative acceptance procedure is implemented for regression-test use, but no supplied run demonstrates acceptance and no inspected consumer installs its verdict into production page admission. See MEM-RTE-page-evaluation. Likewise, CLI mirroring implements one-way server-to-disk delivery and overwrites local drift; it establishes neither server write-back nor a particular later model reader (MEM-RTE-filesystem-mirror).
Learning and self-improvement conclusions must remain separate. The wired trace-fed update and delivery path is a supported contribution; improved capacity for future action remains unestablished. There is no comparison demonstrating that criticism of a consumed theory improved that capacity. For theory-builder conditions 1–4, localized observation/page prose is available, but the inspected account does not establish formulated theory content, decisions depending on that theory, content-directed criticism with blame placement, and retained criticism shaping a next round as four separately satisfied conditions. Structural correction and source withdrawal alone do not supply that account. Reflective self-representation and autonomous theory building are also unestablished; the operation named reflect and automatic maintenance do not establish them. These are bounded unknowns, not whole-system absences. See the epistemic bounded conclusion and WF-CLM-completion-checks.
For the recurring hook pathway across later prompts, durable state and a live advisory channel are wired. Evidence bearing on an independently specifiable improvement objective selecting the update, and subsequent operation causally depending on that change, are not established. Exercised self-improvement, or successful improvement, therefore cannot be attributed at this boundary. Visible model calls are inference, while provider-internal changes and immutable weight identities remain uninspected. See RT-RTE-claude-hooks, RT-BAP-hook-context and RT-CMP-reflect-llm.
Assessment would change with provenance-linked input/output and later-consumer traces, criterion-based production acceptance records, or controlled comparisons of later capacity. Component attribution would require varying the component rather than only comparing whole systems. Full delta/retraction and alternate-consumer inspection could strengthen lifecycle conclusions; deployed enforcement and provider identities would need their own evidence. No target source files were read for this synthesis: its warrants are the supplied accepted records and verification limits.
Limitations
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| Shipped-consumer and working-context inventory is incomplete, including an omitted LangGraph mechanism. The verifier found stable-ID system-message/custom-state delivery, removal of prior memory context on no-memory turns, and retention of latest enabled human/AI text rather than tool messages. | SRC-1; report-verification limit; RT-RTE-claude-hooks, MEM-RTE-recall-selection, MEM-RTE-filesystem-mirror | Named hook, recall and mirror routes; verifier's static LangGraph node inspection, not graph execution | Cross-integration equivalence and complete working-context-control coverage | Canonical records tracing LangGraph selection/replacement and other shipped consumers; execution traces for actual uptake |
| Retained Claude Code progress and its suffix/window selector are not separately classified in the partial profile inventories. Progress commits include post-client-return and empty-content branches. | SRC-1; profile-verification limit; RT-RTE-claude-hooks | Broad integration-local inventory and known hook bookkeeping | Complete per-object, per-admission or per-consumer comparison of hook storage, form, lineage, authority and replay/window control; progress as ingestion success | Separate accepted progress-state and admission/selection records, plus server-completion-linked traces |
| Additional stores, attachments, imports, embedded modes, UI mutations, integrations and evaluators remain incompletely inspected. | SRC-1; boundary; RT-RTE-retain-work, MEM-OBJ-document-text, MEM-RTE-page-maintenance | Complete artifact, partial loop; default SQL and named caller paths | Whole-system inventory, deployment parity, universal SQL storage or complete acceptance/learning coverage | Records tracing material alternate write, maintenance and later-consumer paths and configured-store contracts |
| Source truth and semantic fidelity of extraction, consolidation, refresh, compression and rewriting are not established. | SRC-1; MEM-OBJ-fact-text, EPI-RTE-page-refresh, RT-RTE-context-synthesis, WF-CLM-completion-checks | Static candidate-generation and procedural/structural gates | Lossless reshaping, warranted entailment, produced accepted ampliative claims or answer correctness | Provenance-linked content instances, warranted premises and evidence-consuming checks with named scope/use |
| Complete and timely withdrawal is unresolved beyond local dependency cleanup and guarded refresh. | SRC-1; MEM-RTE-memory-curation, MEM-RTE-page-maintenance, EPI-RTE-source-withdrawal | Inspected PostgreSQL cleanup, partial delta/retraction machinery, best-effort automatic-page nudges | Immediate all-consumer retraction or restoration of every prior derived byte | Complete executor/scheduling records and source-to-page-to-consumer withdrawal traces, including manual pages |
| No supplied operation or causal evidence closes the learning/self-improvement attribution chain; provider state is not frozen by the repository. | SRC-1; RT-RTE-claude-hooks, RT-BAP-hook-context, RT-CMP-reflect-llm, MEM-RTE-page-evaluation | Source-wired trace update, delivery, inference and criterion-relative tests | Observed reliance, improved capacity, exercised self-improvement, benchmark superiority or individual component benefit | Linked update/consumer traces, independent objective and outcome comparisons, component interventions and deployed model identities |
| Policy and effect guarantees depend on configured enforcement and external contracts. | SRC-1; RT-CLM-tenant-check, RT-RTE-extension-load, MEM-BAP-directive-prompt, MEM-RTE-recall-selection, WF-CLM-completion-checks | Named credential bypasses, extension loader, interpreted directives and budget countercases | Universal credential enforcement, extension isolation, semantic directive compliance or hard whole-payload bounds | Deployed admission/isolation contracts and enforced response checks covering alternate and fallback routes |