CocoIndex synthesis

Type: agentic-system-analyses/types/agentic-system-synthesis.md

Bounded synthesis

CocoIndex is an incremental processing and context-indexing framework: application authors declare transformations and target states; the engine schedules work, reuses tracked results and reconciles targets. This account covers the repository at commit 1639793b3602408a9cb220728fb30e306d433473, with an applicability cutoff of 2026-10-15. The whole-system boundary includes shipped drivers, search applications and contributor instructions, not just library APIs. It does not imply exhaustive inspection. SRC-1 supplies implementation and doctrine/design evidence, but no observed run or causal experiment. External databases, provider internals, model weights and the separately linked CocoIndex-code product are outside the frozen source. See the boundary and evidence and Source register.

From authored policy to retained state

The ordinary text-indexing path starts with a CLI or SDK update. Author-written Python fixes source matching, chunking, embedding and schema; the core engine owns scheduling and target reconciliation. Markdown files become chunks, vectors and PostgreSQL rows. Chunk processors receive selected text and target resources, not an open-ended agent task. Awaiting the update completes indexing work rather than producing an answer. Async and blocking Python and the Rust SDK reach the same engine, but CLI checks do not automatically govern SDK calls. These are wired paths, not observed executions. See RT-RTE-index-update.

Two kinds of retained state then affect later invocations. Internal component and function memos preserve results and dependency information in per-app LMDB. The engine automatically substitutes a retained return when identity and validity checks permit reuse. External content rows instead hold material that separate search programs request. Function entries, component memos, caller-assigned user state and ownership/recovery bookkeeping are distinct objects; opaque return bytes cannot be classified from diagnostic summaries. See RT-OBJ-component-memoization, MEM-OBJ-function-memoization, MEM-RTE-function-reuse, MEM-OBJ-state-handle and MEM-OBJ-tracking-info.

Tracked reuse is not universal freshness. Processor, logic, state and provider dependencies constrain memo reuse, but an unchanged model alias need not reveal changed external weights. The embedding wrapper requests inference rather than training and keys its configuration without a resolved weights digest. Text and lineage fields also differ from dense vectors: recipients interpret text, while numerical vectors influence selection. See RT-CMP-sentence-transformer-embedder, MEM-OBJ-doc-embedding and MEM-OBJ-doc-vector.

Maintenance and effect boundaries

Catch-up updates terminate; continuous maintenance additionally requires live application mode. A source configured with live capability alone does not select that mode. The watcher starts before scanning, signals initial readiness and dispatches changes, with configured rescans as a defense against missed events. Readiness is not terminal completion or measured freshness. See RT-RTE-live-maintenance.

Preview withholds planned target application in the core preview branch; it is not a sandbox for arbitrary application code, provider calls or direct I/O. Explicit drop cancels and drains live components, deletes the root and clears per-app storage, with deletion errors propagated. Ordinary orphan cleanup can retain tombstones for retry instead. Drain timeouts and external-store failures prevent an unconditional quiescence or withdrawal guarantee. Internal precommit, sink application and final commit also do not constitute a demonstrated atomic rollback across LMDB and PostgreSQL. See RT-RTE-preview-update, RT-RTE-drop-app and MEM-OBJ-tracking-info.

What reaches a later consumer

Text search embeds an explicit query, orders retained rows by vector distance and prints filename, score and text, with default top-k five. This is requested retrieval, not an unsolicited agent-context injection or a generated answer. Indexing's 2000-character chunk setting and 500 overlap, result count and display truncation are distinct controls, not an established model-window budget. See RT-RTE-query-index and memory core ideas.

The strongest supported trace-fed contribution is the session-search chain: imported conversational logs are automatically filtered into retained textual records and dense retrieval representations, then selected and displayed by a later query invocation. This is wired trace learning under the profile's artifact-and-consumer definition, not demonstrated improvement in future capacity. Invalid JSON, non-text payloads and short text are discarded; displayed results contain only the first 200 characters. Imported context summaries are not shown being generated by CocoIndex or reinstalled as host instructions. See MEM-OBJ-session-embedding, MEM-RTE-session-search and profile verification.

Shipped image APIs and filing search supply additional retained-content consumers. Image responses project filenames and scores; filing search combines semantic and lexical ranks. Their existence defeats an API-only account, but incomplete producer and helper inspection prevents uniform freshness or lineage conclusions across these indexes. See MEM-RTE-alternate-search.

Model-assisted extraction and consequential checks

The representative workflow is manual extraction, selected because one application connects PDF discovery, conversion, a model handoff, validation and database delivery. It is a fixed model-assisted pipeline, not a sub-agent coordination loop. The operator starts processing; file processors convert PDF bytes to Markdown; the provider receives that Markdown and a source-support instruction with a response schema. The converter's CPU setting is distinct from its GPU-runner placement. Code validates the returned structure, constructs a row and delegates SQL effects to reconciliation. Successful conversion, inference, validation and database writes are conditions of delivery. A conversion or extraction exception prevents this invocation's new row declaration, not necessarily retention of an old row or effects from other files. No later manual-record answering consumer was traced. See RT-RTE-manual-extraction and the workflow account.

The request to use only supported content is not a factual acceptance gate. Pydantic checks shape without comparing assertions to the manual. List counts are exact cardinalities of the extracted lists, not independently verified counts of the source API. Without actual paired inputs, outputs and semantic assessment, extraction's content relation remains indeterminate rather than automatically classified as derivation or conjecture. See EPI-CLM-supported-extraction, EPI-RTE-module-check and EPI-RTE-module-count.

Entity resolution adds a different operative check. Invalid candidate identifiers produce corrective request messages and retries; exhaustion of that invalid-output retry cap returns an empty decision, and the generic engine rejects self-matches or out-of-list identifiers. Valid decisions affect canonical-map operations under existing-canonical policy, which can override model preference. These checks constrain interface validity, not identity truth. Contributor type/build checks likewise concern their own domains and emit host signals, not factual endorsement or a proven production veto. See EPI-RTE-pair-check, EPI-RTE-pair-disposition, EPI-RTE-contributor-check and EPI-RTE-hook-result.

Assessment and what would change it

For maintaining and searching authored source context, the records establish implemented transformation, reuse, maintenance and delivery routes. They do not establish the unconditional reliability or absence of context gaps in the public fresh-context claim. For source-faithful extraction, shape admission and operational retention are supported; evidence-consuming factual acceptance and subsequent integration of accepted claims are not established on the inspected manual route. Neither result is a system-wide epistemic grade. See EPI-CLM-fresh-context and the epistemic bounded conclusion.

Local corrective retries support request-context adaptation, while the session path supports durable trace-derived retrieval context. Whether criticism of a consumed theory improves future capacity remains unresolved: neither mechanism establishes that result. The accepted runtime supplies no condition-by-condition formulated-theory assessment, so theory-builder conditions 1, 2, 3 and 4 remain individually unestablished here, not evidenced absent. Reflection is independently unestablished. The inspected workflows use author-fixed progression rather than autonomous task selection; autonomous theory construction is independently unestablished. Self-improvement of CocoIndex's own behavior-determining organization is also unestablished: indexed outputs and local retries do not show an objective-responsive organizational update entering later operation with causal dependence. These are bounded non-attributions, not repository-wide negatives. See the runtime account and epistemic bounded conclusion.

An inspectable update-to-query trace would change the operation assessment; manual/output comparisons with an evidence-consuming support check would change the extraction-warrant assessment. Exact model identities and tested external change contracts would sharpen freshness claims. Tracing omitted graph and alternate producers would improve route comparisons. Demonstrating improved future capacity would additionally require a retained change, its later consumer and an appropriate comparison; attributing self-improvement would require objective-responsive change to the system's own organization and subsequent operation causally dependent on it. None is supplied by this run. No additional source reads were made for this synthesis.

Limitations

Limitation Affected SRC IDs and record citations Inspected boundary Conclusion prevented Evidence that would resolve it
Unresolved pair-retention attribution and local dependency-reference defect SRC-1; EPI-RTE-pair-proposal, RT-OBJ-component-memoization, MEM-OBJ-function-memoization, MEM-RTE-function-reuse; report-verification Limits Epistemic ledger attributes retention to the component object; the resolver decorator identifies a memoized function, while memory distinguishes function and component schemas. A unified, precise attribution of pair-return persistence to the ledger's component object. Memoized-return affordance and separate internal object classes remain supported. Corrected declaring-member dependency tracing the decorated resolver through its actual persistence mechanism.
Graph route inventory is partial SRC-1; EPI-RTE-pair-disposition; memory coverage; report-verification Limits Reports end at returned canonical maps; verification identifies preceding transcript extraction/filtering and shipped graph declarations consuming those maps. Complete graph admission, lineage-preservation and downstream-authority comparisons; no complete acceptance or absence verdict follows. Analyst records covering the omitted extraction, filtering and canonical-map-to-node/relationship chain.
Alternate producers and later content readers remain untraced SRC-1; MEM-RTE-alternate-search, MEM-OBJ-session-metadata, RT-RTE-manual-extraction; report-verification Limits Shipped image/filing projections, metadata writes and conditional manual delivery are inspected, not all producer/helper or later-reader paths. Uniform all-index freshness, full metadata consumption and downstream answering-agent use. Producer, selector, retirement and later-consumer records for each unresolved branch, with execution evidence for operation claims.
Imported checkpoint provenance is unresolved SRC-1; MEM-OBJ-session-embedding, MEM-RTE-session-search; report-verification Limits Local parsing, embedding and snippet display; upstream summary production and host reload are outside the inspected chain. Claims that CocoIndex generated the summary, preserved its reasons or retained restorable host state. Frozen upstream generation and host-consumption evidence, including what the summary omits and retains.
Implementation is not observed operation or benefit SRC-1; Source register, RT-RTE-index-update, RT-RTE-query-index, MEM-RTE-session-search Single frozen repository; no target, test, provider or database execution evidence. Exercised freshness, successful withdrawal, extraction accuracy, retrieval benefit or improved future capacity. Inspectable bounded execution artifacts and evaluations; controlled comparisons with appropriate variation for causal attribution.
External identity and effect contracts remain conditional SRC-1; RT-CMP-sentence-transformer-embedder, RT-CMP-llm-model, RT-RTE-drop-app, MEM-OBJ-tracking-info Adapter-facing code, configuration keys, sink lifecycle and cleanup protocol; external implementations are excluded. Exact model behavior, universal alias-drift detection, unconditional cleanup or atomic cross-store rollback. Pinned model identities and external contracts, plus failure and recovery evidence across all affected stores.
Opaque payloads and other included routes prevent complete enumeration SRC-1; MEM-OBJ-state-handle, MEM-OBJ-function-memoization, memory coverage; profile verification Generic state/memo APIs and selected examples; remaining connectors, retained outputs and consumers are not fully traced. Complete memory-value comparisons or source-wide negatives about learning, autonomy, reflection, theory construction or acceptance. All ten profile axes remain partial. Part-specific payload, admission, derivation and consumer records across the unresolved inventory; property-specific evidence rather than inference from another property.