Semantic Engine synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From selected sources to retained retrieval state
Semantic Engine is an operator-driven document memory pipeline, not an autonomous agent runtime. This account covers its shipped ingestion, storage, embedding, retrieval, visualization and export responsibilities at commit 7e2481671cad01f0b4c89f2a1877d6733d7e552c, with an analysis cutoff of 2026-10-17. SRC-1 supplies inspected implementation and declared design only. No target code, tests, services or model calls were executed, and no execution trace or causal experiment was supplied. The whole-system boundary describes functional source coverage, not a demonstrated integrated deployment (boundary, Source register).
The declared work is ingesting, embedding and visualizing documents and their relationships. Its most concrete connected workflow is the CLI quickstart: a human selects an archive corpus, downloads it, loads documents, embeds chunks and separately submits a query. Files and database rows connect the steps; the operator owns cross-command continuation and recovery. Fixed scripts choose the next computational steps. Models return numerical embeddings rather than selecting tools or delegating reasoning. Terminal document output completes this workflow, not reviewer acceptance or answer generation (RT-CLM-pipeline-purpose, workflow account).
Downloaded archive files are raw input, not automatically the searchable corpus. Existing local files are skipped without a refresh check. The archive loader filters retweets and replies to others, joins surviving self-threads and drops orphan replies. The alternate RSS loader converts HTML to titled Markdown under different field checks; archive filtering does not govern RSS. Both automatically admit rows after human source selection, rather than obtaining per-item approval. Fresh IDs and stored hashes do not prevent duplicate reimports. Separate root and chunk insertions lack a spanning driver transaction, so interrupted ingestion need not leave a complete searchable population (MEM-OBJ-archives, RT-RTE-ingest-documents).
The dataset-selected SQLite file separates full root text, chunk text and offsets, provenance metadata, and dense chunk vectors. Segmentation preserves an ordered text collection but need not preserve a complete assertion in each fragment; its 512-JavaScript-string-unit size is not a model-token guarantee. Metadata preserves associations rather than verified trust: RSS author and date fallbacks can substitute feed labels and ingestion time. These distinctions matter because chunks become model inputs while roots and labels become human-facing output (MEM-OBJ-docs-content, MEM-OBJ-chunks-content, MEM-OBJ-docs-metadata, EPI-RTE-split-text, EPI-RTE-source-reshaping).
Derivation, later selection and maintenance
The embedding driver reads null-vector chunks, sends their text to the configured LMStudio model and constructs ID-keyed updates. Admission checks for a nonempty returned array, not correspondence, dimensions or semantic quality. A failed batch advances to the next batch; a fresh invocation can revisit still-null rows. Non-null selection is therefore neither eventual completion nor freshness checking. The external model has no immutable weights identity recorded with each vector. Changing the configured model does not invalidate older vectors (RT-RTE-embed-chunks, RT-CMP-lmstudio).
The strongest supported durable contribution is the implemented path from imported text to retained numerical ranking inputs: later query selection depends on those values. That is not a provider-weight update or demonstrated learning benefit. Even successful vector normalization remains unobserved: the schema declares F32 blobs, while the update constructs JSON-stringified CASE values without the adapter's ordinary conversion expression (MEM-OBJ-chunks-embedding, RT-BAP-vector-ranking).
A query sends only its current text to LMStudio. SQL then ranks non-null stored chunk vectors by cosine similarity, takes at most fifty candidates for the ten-root limit, deduplicates parent IDs and reloads full roots for printing. Repeated chunks from a few roots can consume the window, so ten distinct results and the globally best ten roots are not guaranteed. This is requested read-back to a terminal user, not unsolicited context injection or retrieval into a generative answer model. The quickstart's “RAG” label establishes no missing generation stage (RT-RTE-query-documents, EPI-CLM-rag-quickstart).
The operator can clear all non-null vectors in a dataset. That removes retrieval eligibility and enables recomputation without removing roots, chunks or provenance. It keeps no former vector version and does not synchronize derived map or Atlas copies. This is explicit maintenance, not evidence-responsive factual retraction or automatic model-version migration (RT-RTE-clear-embeddings).
What the other outputs and checks establish
Canvas projects vectors through PCA and affords a GeoJSON/map consumer, but its fixed database path, string-vector assumptions, missing null exclusion and generated/requested filename mismatch prevent claiming a connected current-schema route. Joined content columns also leave displayed text identity ambiguous. Atlas export has independent initialization and obsolete-schema obstacles, plus ID and array-shape qualifications. It explicitly requests topic modeling; neither its output nor its remote acceptance and later consumers are available. These alternatives remain in scope without being joined into the terminal workflow (MEM-RTE-canvas-display, MEM-RTE-atlas-export, EPI-RTE-atlas-topics).
The separate Knowledge Gate page sends two current texts to OpenAI and displays their cosine similarity; it does not gate corpus admission. Equal-length and nonzero-magnitude guards enforce limited arithmetic preconditions, not factual truth or semantic equivalence. Authored chunker tests concern fixture behavior, not retrieval relevance, and have no inspected ingestion-acceptance consumer. Optional clustering groups copied content; its processor call is commented out, and SSE diagnostics do not select an accepted factual explanation (RT-RTE-compare-texts, EPI-RTE-cosine-guards, EPI-RTE-chunk-tests, EPI-RTE-cluster-items).
Scenario-relative assessment
For an operator seeking source documents by similarity, the source establishes acquisition, durable selection state and full-document terminal delivery, subject to provider/database compatibility. It does not establish relevance, reliability or successful execution. For factual knowledge acceptance, the inspected local routes provide no evidence-consuming decision against a factual criterion for a named use and scope. Storage, ranking and display grant operational influence without adding that warrant; this bounded absence does not cover undocumented human review or opaque providers (RT-RTE-query-documents, EPI-ABS-claim-acceptance).
The retained-vector contribution leaves improved capacity for future action unresolved: there is no quality comparison or criticism-driven update showing improvement. Trace learning likewise has no established qualifying trace-fed write in the accepted records, but that is not a system-wide negative. Initial vectors and external processing remain unresolved (RT-BAP-vector-ranking, MEM-OBJ-chunks-embedding, profile verification).
The runtime does not establish a formulated-theory revision route, so theory-builder conditions 1–4 are not separately determined here; numerical derivation must not substitute for those findings. Criticism of a consumed theory improving future capacity is unestablished. Reflection is independently unestablished: these records do not show a self-representation with both causal directions. Autonomous end-to-end progression is not established by automatically executing individual scripts; the workflow requires operator continuation. Self-improvement is also unestablished over the inspected cross-invocation horizon: retained vectors can shape later behavior, but no improvement objective, objective-bearing evidence determining an update, or exercised evidence-responsive self-change chain is established. These are independent limits, not a bundled grade (RT-RTE-embed-chunks, RT-BAP-operator-config, RT-BAP-vector-ranking, workflow account).
A registered end-to-end trace inspecting stored vectors and later selection would change the operational assessment. Relevance judgments and controlled comparisons would be needed for benefit or component attribution. Repaired canvas/export paths plus traced downstream consumption would change their connection status. Evidence-consuming factual decisions, or a traced criticism-to-retained-update-to-later-action pathway, would change the corresponding acceptance or improvement assessment rather than merely adding more retained data.
Limitations
Both supplied verifications declare no blockers or publication limits. The following evidence bounds still prevent stronger conclusions; no unresolved opposed finding was supplied.
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| No execution or quality comparison | SRC-1; RT-RTE-ingest-documents, RT-RTE-embed-chunks, RT-RTE-query-documents | Frozen CLI implementation and operating instructions; no registered runs | Successful completion, relevance, reliability, latency or causal benefit | Registered traces, relevance references and appropriately controlled comparisons |
| External model identity and corpus warrant unresolved | SRC-1; RT-CMP-lmstudio, RT-CMP-openai, MEM-OBJ-docs-content, MEM-OBJ-chunks-embedding | Caller/configuration code, not provider internals, actual corpus or initial vector provenance | Immutable model behavior, cross-call compatibility, acquired claim truth or complete lineage | Versioned provider identities, corpus/provenance evidence and compatibility checks |
| Binary vector normalization unobserved | SRC-1; MEM-OBJ-chunks-embedding | Declared vector schema and JSON-valued CASE update | Guaranteed interoperable embedding storage and query execution | Trace of actual stored representation and successful current-schema query consumption |
| Canvas connection and text identity unresolved | SRC-1; MEM-RTE-canvas-display, MEM-OBJ-geojson | Processor, PCA and browser consumer with database, parsing, column and filename mismatches | Connected current-corpus display or proven root-versus-chunk labels | Aligned paths/encoding, explicit text selection and traced generation/reload |
| Atlas reachability and remote semantics unresolved | SRC-1; MEM-RTE-atlas-export, EPI-RTE-atlas-topics | Local exporter/ORM; remote payload handling and later consumers uninspected | Successful export, retained topics, new accepted claims, complete remote memory or read-back classification | Repaired initialization/schema/ID/shape handling and registered remote outputs, storage and consumer evidence |
| Local factual-acceptance absence does not cover external processes | SRC-1; EPI-ABS-claim-acceptance, EPI-RTE-atlas-topics | Recorded local admission, checking and consumption search; excludes undocumented human review and opaque providers | Universal absence of accepted knowledge or external acceptance | Evidence of those external decisions, their criteria, scope and consequential consumers |
| Durable ranking is not an established improvement or theory loop | SRC-1; RT-BAP-vector-ranking, RT-BAP-operator-config, RT-RTE-embed-chunks, profile verification | Imported-document derivation and later ranking; unresolved initialization and external processing | Improved future capacity, trace-learning yes/no, separately determined theory-builder conditions, reflection or exercised self-improvement | Property-specific records establishing trace origin, formulated criticism, retained consequences, self-representation or objective-responsive later operation, with benefit comparisons where claimed |