semquery synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From documents to reusable retrieval state
Semquery is a local document-search and question-answering system. Its shipped semq CLI and Rust Engine own collection registration, indexing, maintenance, retrieval and answer return. It is not an enclosing agent runtime: code chooses the stages, while the chat model generates answer text rather than selecting tools or delegating tasks. This account covers the whole shipped functional system at commit 0820927160b1400990fa8cc057807c73bbf6d025, with an applicability cutoff of 2026-11-09. The evidence is SRC-1 implementation and attributed design/documentation, not an observed execution or causal experiment. External document authorship, dependency internals, downloaded model weights and downstream hosts are not frozen evidence (boundary, Source register).
A user ordinarily registers a file or directory with semq add. Registration persists before model loading and indexing, so it can survive an indexing failure. Software then selects eligible files, extracts text, slices it into chunks, derives lexical terms and embeddings, and commits retrieval state. Starting this operation is not approval of every extracted claim: readability, hashing and storage success govern admission, not a truth check or per-chunk human review (RT-RTE-index-admission, EPI-RTE-reader-import).
The durable corpus has distinct parts. SQLite stores natural-language chunk text, FTS5 lexical derivatives, sqlite-vec dense vectors and symbolic ownership/freshness metadata. The vectors are representations of document text, not the embedding model's weights. Later consumers use chunk text as reference context and reranker input, lexical terms and vectors as ranking inputs, and hashes, associations and baselines as maintenance controls. Persistence spans workspace reopen; it does not establish source truth or successful later reliance (MEM-OBJ-chunks, MEM-OBJ-fts-chunks, MEM-OBJ-vec-chunks, MEM-OBJ-document-paths).
Content-hash reuse deduplicates identical chunk text and preserves links to multiple documents. That economy has a provenance cost: a shared chunk retains one byte range, while read-back independently selects one associated document. The displayed document and range need not identify the same occurrence. PDF/DOCX ranges also address extracted strings rather than original binary-file offsets. Extraction itself can lose structure: the inspected DOCX reader can concatenate text from adjacent paragraphs without a separator. These are implementation-supported countercases, not observed bad citations (MEM-CLM-chunk-lineage, EPI-RTE-reader-import).
From a question to model-visible context
The representative workflow is semq ask over an indexed workspace. Engine reopens storage and lazily loads embedding, reranking and chat components. The retriever concurrently performs segmented/sanitized keyword recall and raw-query embedding/vector recall. Either recall error stops the route rather than yielding a one-channel fallback. Reciprocal-rank fusion combines positions, and an optional reranker scores a configured prefix before final selection. Injected components can omit reranking or replace other default behavior; these extension points are not an authorization gate (RT-RTE-hybrid-search, RT-RTE-config-admission).
Ask requests five hits. Code formats their text, paths and ranges into numbered context blocks with the question and a grounding/citation instruction. This is request-driven read-back: automatic selection fulfills a search or answer request, rather than independently pushing memory or initiating a task. Relevance ranking determines what can influence an answer; it supplies no truth or entailment license (EPI-OBJ-prompt, RT-RTE-hybrid-search).
The offered context is not necessarily the context inference receives. Both concrete GGUF completion methods reserve generation room and retain only the suffix of an over-budget formatted token sequence. Leading instructions, the system message or citation passages can disappear while the question near the end survives. No summary or within-generation reload restores them. Each completion uses a fresh context; another retrieval invocation can reach retained corpus material again, but the model has no dereference operation for discarded tokens in this call. Custom LLM implementations have their own, uninspected context policies (RT-RTE-answer-generation, RT-CMP-gguf-llm).
From generation to delivered output
With no hits, synthesis returns an empty answer without generation, although ordinary Engine model setup already occurred. With hits, the model generates raw answer text under a natural-language grounding request. Retained text and configured system guidance have model-interpreted influence, not an enforced factual constraint. A generated proposition could repeat, derive from or exceed its input; without an actual answer and its actual model-visible inputs, that relation remains indeterminate (RT-RTE-answer-generation, RT-BAP-prompt-influence, EPI-OBJ-answer).
Citation checking has a narrower, concrete effect. Numeric marker membership and deduplication control the citation metadata vector. They do not check whether a sentence follows from a passage, require a citation, or remove invalid markers from raw answer text. The allowed marker set comes from pre-truncation hits, so even a valid marker does not prove the passage reached inference. This implements the README's source-linked answer interface without establishing factual acceptance of answer propositions (EPI-RTE-marker-check, RT-CLM-citation-filter, EPI-CLM-cited-answers).
Text-mode CLI output prints token deltas before citation checking and does not replace them with terminal text. JSON prefers the terminal Answer; if the stream ends without it and without an error, the CLI can return buffered text with no citations. Collecting Engine::ask instead rejects a missing terminal answer. Token-send failures do not necessarily interrupt inference; best-effort callback delivery applies on current-thread Tokio, not unconditionally to the ordinary multithreaded CLI. Errors propagate outward, possibly after partial text display, and recovery is caller reinvocation rather than automatic retry or durable resume (workflow delivery, RT-RTE-answer-generation).
Maintenance and what this architecture establishes
For repeated questions over operator-selected documents, the supported contribution is durable, query-selected access to source material and a local model-backed answer channel. It is not demonstrated improvement in future capacity. Answers are not written back by the inspected synthesis route; diagnostic logs remain raw retention without an established automatic policy-update consumer. External saved configuration can change subsequent controls or system guidance, but its replacement is not an implemented evaluation-fed self-revision loop. Neither fixed inference parameters nor these route findings license a system-wide trace-learning negative across opaque alternatives (MEM-OBJ-semq-log, RT-RTE-answer-generation, RT-RTE-config-admission, profile verification).
Theory-builder conditions 1–4 are not separately established by the supplied runtime records. Nor do the records establish that criticism of a consumed theory improved future capacity, a reflective system, an autonomous theory builder or self-improvement at this boundary. These are independent unestablished properties, not consequences of a single negative judgment. The implemented automatic indexing and returning answer stages remain supported without treating them as autonomous theory construction or evidence-responsive organizational change (RT-RTE-index-admission, RT-RTE-answer-generation, RT-RTE-config-admission).
Maintenance changes availability rather than reliance after a falsity finding. Explicit deletion and orphan cleanup can forget solely owned indexed material while preserving shared chunks; they do not delete source files, earlier outputs or model weights. Directory sweep uses a different predicate from explicit collection deletion and has a sibling-prefix hazard. Empty or invalid-content transitions can leave old chunks searchable. Add-time config drift is also not a global freshness barrier: search/ask do not require reindexing, and successive source passes can clear the shared drift baseline before all sources reembed. Transactions protect individual uncommitted batches, not whole indexing invocations (MEM-CLM-empty-retention, memory maintenance annotation, RT-RTE-index-admission, RT-CLM-transaction-scope).
These qualifications matter most where an application needs exact current-file mirroring, precise occurrence provenance or verified answers. The present evidence supports none of those guarantees. Per-occurrence provenance, explicit empty-content withdrawal, path-safe sweep predicates, whole-workspace baseline handling and a model-visible-context-aware support check would change the respective architectural assessments. Pinned artifacts and inspectable end-to-end traces would resolve different questions about deployed identity and operation; controlled outcome comparisons would still be needed to establish retrieval benefit or causal improvement. No new source inspection was performed for this synthesis.
Limitations
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| No observed run or causal comparison | SRC-1; RT-RTE-hybrid-search, RT-RTE-answer-generation | Shipped implementation, documentation and test/driver source, not executed outcomes | Successful operation, answer quality, compliance, delivery or causal benefit | Traces tying actual inputs, selected hits, model-visible tokens and outputs to the frozen implementation; controlled outcome comparisons for benefit |
| Extraction and source warrant are not certified | SRC-1; EPI-RTE-reader-import, MEM-OBJ-chunks | Text/PDF/DOCX extraction and chunking; dependency internals and user files unsupplied | General semantic preservation or endorsement of imported claims | Original/extracted document comparisons, dependency evidence and claim-specific source assessment; boundary-preserving DOCX extraction would remove the named countercase |
| Shared-text provenance is not per occurrence | SRC-1; MEM-CLM-chunk-lineage | Chunk insertion, association lookup and citation display | Exact original-file location from a displayed path/range | Occurrence-specific ranges tied through read-back to the selected document, plus format-aware source mapping |
| Index maintenance does not guarantee current-file mirroring | SRC-1; MEM-CLM-empty-retention, RT-RTE-index-admission | Empty/invalid-data skips, shared config baseline, sequential source indexing and query consumers | Guaranteed empty-content withdrawal, complete all-source reembedding or query-time freshness | Explicit withdrawal on skipped changed content; source-scoped or completion-scoped baselines and freshness enforcement with corresponding traces |
| Transaction and cleanup guarantees are path-scoped | SRC-1; RT-CLM-transaction-scope, RT-RTE-index-admission, memory maintenance annotation | Batch commits, explicit deletion, orphan cleanup and raw-prefix directory sweep | Whole-run rollback, demonstrated shared-connection isolation or universal sibling-safe cleanup | Whole-run recovery/isolation protocol and path-component/surviving-root-safe sweep implementation; failure/concurrency evidence |
| Exact lexical replacement is unresolved; carried report-verification limit | SRC-1; MEM-OBJ-fts-chunks, report-verification limit | FTS5 schema and reembedding insert chain: nonunique chunk_id with INSERT OR REPLACE and no prior lexical-row deletion | Exact-replacement and duplicate-free lexical-ranking comparisons; segmentation, persistence and read-back remain supported | Explicit unique/replacement semantics or deletion-before-insert, with retained row-level maintenance evidence |
| Offered hits can differ from model-visible context | SRC-1; RT-RTE-answer-generation, RT-CMP-gguf-llm | Both default GGUF completion methods; custom policies unknown | Complete evidence delivery or inference visibility from citation membership | Retained actual token inputs and visibility-aware budgeting/marker construction |
| Citation metadata does not validate assertions | SRC-1; EPI-RTE-marker-check, RT-CLM-citation-filter | Direct/streamed synthesis and CLI output | Sentence support, answer truth or raw-text cleanup from valid metadata | Proposition-level evidence-consuming evaluation and a consequential answer admission/revision gate; answer/input instances for bounded support findings |
| Deployed artifacts and injected behavior remain unresolved | SRC-1; RT-CMP-fast-embed, RT-CMP-fast-reranker, RT-CMP-gguf-llm, RT-RTE-config-admission | Default adapter integration; actual weights/tokenizer contents, dependency internals and downstream hosts not supplied | Immutable inference identity or universal guarantees across injected implementations | Authenticated exact artifacts and dependency/host evidence; adapter revision forwarding and verification where applicable |
| External admission and improvement paths are not established | SRC-1; MEM-OBJ-config, MEM-OBJ-semq-log, RT-RTE-config-admission, profile verification | Known software admission and read-back; external authorship, replacements and opaque alternatives unresolved | Per-content human approval, complete memory inventory, system-wide trace-learning yes/no or improved capacity from retention | Admission provenance; a qualifying automatic trace-to-durable-update-to-later-consumer chain or bounded absence warrant; separate future-capacity evidence |
| Stream completion and visible delivery differ | SRC-1; RT-RTE-answer-generation, workflow delivery | Token callback, Engine forwarding and CLI text/JSON consumers | Unconditional token delivery, immediate inference cancellation or durable answer resume | Runtime-specific delivery/cancellation traces and an implemented interruption/checkpoint protocol for stronger guarantees |