AI Research OS Workshop synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
A research-memory plugin, not an enclosing runtime
AI Research OS Workshop preserves topic research as a local Markdown wiki and canonical YAML catalog. This account covers its shipped plugin, prompts, connectors' usage instructions and Python helpers at dc6660555266239cd36483cf51b5ab39febacf02, with cutoff 2026-10-12. It is a whole-system boundary for that plugin, not for Claude Code, Codex or the external source services. The registered repository supplies implementation and doctrine/design evidence. No supplied execution trace or causal experiment establishes successful operation, factual quality or benefit. Generated examples were not treated as execution verification. See the boundary and evidence and Source register.
The host owns inference, agent dispatch, tools and the enclosing session. Natural-language skill instructions ask its orchestrator to select sources, delegate writing, join outputs and return results. Python helpers own narrower symbolic transformations. The plugin does not supply an inspected scheduler or deployment permission envelope. Model versions, parameter updates and endpoint substitution remain uninspected in the host and NotebookLM services. Static skill guidance, accumulated research context and executable effects are separate authority channels: RT-BAP-skill-guidance, MEM-BAP-retained-knowledge, RT-BAP-helper-execution, RT-CMP-host-model and RT-CMP-notebooklm-model.
From request to retained research
The representative workflow adds explicit web URLs to an existing research directory without discovery. The user supplies topic and source identities; the orchestrator checks layout and prior catalog identities, shows a plan, and normally proceeds unless confirmation was requested. Missing canonical YAML blocks mutation while permitting a stricter read-only query fallback. These are interpreted policy gates, not intercepted tool permissions. The user authorizes the route, not every generated proposition. See RT-RTE-append-ingest and the workflow account.
The orchestrator fetches and strips web text, with WebFetch fallback. A builder receives seed JSON and raw-only instructions, persists source content and returns compact metadata. One source writer per successful raw source writes a source page. The orchestrator joins their reports and delegates entity/concept pages when current-run mentions reach two distinct sources. Those writers aggregate assigned source pages rather than raw files. Overview and thesis writers then discover selected wiki content under a five-full-page budget. The driver finally augments pointers, builds YAML with prior-index carry-forward, regenerates navigation, logs the operation, cleans scratch JSON and reports counts, paths and skips. Lint is suggested afterward, not a mandatory final acceptance gate. Local fetch retries and skips permit partial progress; no multi-file rollback or complete semantic-worker retry protocol is established. See RT-RTE-append-ingest, RT-RTE-index-build and workflow annotations.
The layers preserve different material. Raw files contain imported or extracted sources, not necessarily original bytes. Manual highlights are a separate optional layer; some sources have highlights but no full text. Source pages compress individual sources, topic pages aggregate them, and wiki/synthesis.md is an argumentative thesis with assumptions, supporting moves, counter-evidence and revision conditions. Catalog strings supply summaries; identities, scores and URIs supply access and selection structure. Research files therefore combine interpreted prose with fixed symbolic fields; generated charts add executable data snapshots, while visual payloads remain separately qualified. See MEM-OBJ-raw-documents, MEM-OBJ-key-highlights, MEM-OBJ-source-pages, MEM-OBJ-wiki-pages, MEM-OBJ-synthesis, MEM-OBJ-index-summary and MEM-OBJ-index-metadata.
Model writers select and admit claims without routine per-claim human approval. Optional discussion changes propagation after source pages already exist; it need not withdraw those sources. Deliberate human KEEP blocks request preservation in later eligible rewrites, but do not enforce a filesystem veto. Catalog persistence is wired code: seed scores become 1.0, supplied prior rows survive unless superseded, and navigation uses both YAML and the live wiki tree. These guarantees depend on the actual inputs. Omitting the prior index loses carry-forward; seed precedence is not complete deduplication; missing pointers are projected rather than validated. See MEM-BAP-keep-preservation and RT-CLM-index-contract.
Deep discovery is an alternate admission route, not unrestricted autonomous research. A user selects a bounded preset; researchers filter and cap hits from available Obsidian, Readwise and NotebookLM sources. Fixed dedup retains the longest summary per identity, while a gap analyzer uses metadata to propose successor queries. Neither relevance, summary richness nor seed priority measures truth. GitHub ingestion instead uses a reusable mutable clone and model-written architecture/module specifications, with one catalog seed per repository commit; curated specifications are not a complete retained code snapshot. See RT-RTE-deep-discovery and RT-RTE-github-ingest.
Later consumption and criticism
A subsequent query selects catalog summaries and follows source-page, highlight, full-text or wiki pointers only as needed. This is requested progressive read-back, not a demonstrated activation trace. Within an ingest, assigning a matching prior page to a delegated writer is a distinct upstream push operation; summary writers and lint judges instead request their own selected inputs. No independent unsolicited-memory scheduler is established by those routes. Query bookmarks or answers citing at least two sources can trigger save-back, but its knowledge-document landing recipe conflicts with the authoritative question-page-only addition map. The missing-YAML fallback disables saves, logging and navigation writes. See RT-RTE-query-answer, RT-CLM-query-safety, MEM-OBJ-question-pages and profile verification.
Lint offers two different checks. Fixed programs derive structural findings about recognized body wikilinks, catalog pointers, link counts and target-file existence. They do not verify headings, ordinary Markdown citations or claim support. Model judges inspect bounded wiki content for contradictions, stale assertions and research gaps. Contradictions and questions append to rolling files; stale assertions are flags for user judgment, not automatic withdrawals. The summary writer can later read retained criticism, but changed reliance or correction is unobserved. See EPI-RTE-lint-mechanical, RT-RTE-lint-maintenance, MEM-OBJ-contradictions and MEM-OBJ-open-questions.
Source links and synthesis markers make intended attribution inspectable. They do not authenticate imported content or check summary fidelity. Thesis confidence uses source counts and agreement, not independently calibrated warrant. A comparison writer can refuse missing subjects or thin support, but routine integrated dispatch is unresolved. A two-source save trigger licenses recall, not correctness. No supplied run identifies a produced ampliative proposition, its premises and an evidence-consuming acceptance decision for a named use. Semantic relations of model outputs therefore remain indeterminate rather than being inferred from the word “synthesis.” Retention and operational consumption are not established lifecycle integration after acceptance. See EPI-CLM-source-attribution, EPI-CLM-synthesis-confidence, MEM-RTE-comparison-writing and the epistemic conclusion.
Distill selects research relative to a supplied writing corpus, not seed importance, and exports a compact appendix for a documented later writer role. Actual delivery is unobserved. Render delegates selected wiki pages to slides, brief, canvas or chart writers. Charts retain Python with embedded numerical constants and request host execution without separate program-content approval. Re-execution uses that snapshot, not fresh wiki data. A prompt/path-only noop gate can preserve an older render after source bodies change. Neither executable success nor reproducible pixels would certify the selected data. See RT-RTE-distill-export, MEM-OBJ-research-appendix, RT-RTE-render-output and MEM-OBJ-chart-script.
What this establishes for research across sessions
The strongest supported contribution is a designed chain from selected sources to retained, inspectable thesis and criticism, with later consumption affordances and wired catalog persistence. It gives subsequent work places to recover an argument and its objections rather than requiring a fresh source search. It does not establish improved capacity for future action: there is no retained criticism-to-revision-to-capacity comparison. Original conversations are outside the local research artifacts; operation summaries do not establish automatic trace-fed learning. The trace-learning and trace-source classifications remain not determinable, not system-wide negatives. See RT-RTE-append-ingest, RT-RTE-lint-maintenance, MEM-OBJ-log and profile verification.
The runtime supports separate afforded, not observed, theory-builder conditions: (1) a localized whole thesis; (2) requested content-dependent consumption; (3) counter-evidence guidance requesting weakening or splitting; and (4) retained criticism eligible to shape another ingest. Condition 3 is not an observed challenged claim, placement of blame and resulting revision, and condition 4 is not exercised iteration. Headings make assumptions and argumentative moves inspectable and revisable, without a fine-grained semantic revision API. Criticism of consumed theory improving future capacity remains unestablished independently of these four affordances. See RT-RTE-append-ingest and RT-RTE-lint-maintenance.
Reflection is also not established: these routes represent external research, not demonstrated bidirectional self-representation of the plugin's operations. Computational writing and selection coexist with optional human redirection and human stale-claim resolution; a complete exercised autonomous theory cycle is not established. Exercised self-improvement at the plugin boundary is separately unestablished: the records show research-product updates but do not establish an evidence-responsive change to its own organization followed by later operation causally depending on that change. This does not classify every retained output as necessarily external to organization, nor deny a dormant pathway; it withholds the stronger claim. See RT-RTE-append-ingest, RT-RTE-lint-maintenance and the epistemic conclusion.
For explicit-source research followed by a later question, the supported architecture is layered persistence and selective reuse, not an independently accepted knowledge pipeline. For accumulated cross-run synthesis, current-run propagation and bounded reads prevent a completeness guarantee. For an audit, structural predicates and advisory criticism must remain distinct. Assessment would change with inspectable handoff traces, paired source/output fidelity checks, criterion-based acceptance decisions, retained criticism and resulting changed reliance, and later capacity evidence. Corrected pointer schemas, catalog-before-summary ordering, failure filtering, consistent query permissions, isolated commit caches and content-sensitive render refresh would address specific source seams; their presence alone would still not demonstrate benefit.
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No execution or causal comparison supplied | SRC-1; RT-RTE-append-ingest, RT-RTE-query-answer; boundary | Frozen plugin source; examples not used as execution verification | Successful dispatch, activation, factual quality, latency savings and improved capacity | Provenance-linked end-to-end traces and appropriately controlled outcome/capacity comparisons |
| Host and service internals excluded | SRC-1; RT-CMP-host-model, RT-CMP-notebooklm-model | Plugin requests, not enclosing runtimes or services | Model fixity, provider learning, host retention, permissions and deployed confinement | Pinned host/service contracts and implementation or execution evidence under a separately authorized boundary |
| Lint-result parts lack separate record identities; unresolved reconciliation issue carried by report verification | SRC-1; EPI-OBJ-lint-findings, MEM-OBJ-log, MEM-OBJ-contradictions, MEM-OBJ-open-questions; report limits | Structural JSON, report-only stale judgments, rolling semantic findings and durable/transient log container | Fully part-addressed evaluator, storage and retention comparisons; containers cannot be treated as homogeneous memory mechanisms | Analyst-declared diagnostic, stale-result and durable/transient parts with their distinct consumers and retention |
| Raw-path and augmented-JSON handoffs are inconsistent; failed seeds are not explicitly filtered | SRC-1; RT-RTE-append-ingest, RT-RTE-index-build; workflow annotations | Builder URI return, driver raw_path input and original-array augmentation, catalog normalization | Reliable raw-pointer delivery or exclusion of failed sources from catalog | Consistent handoff schemas, explicit success filtering and traceable resolved pointers |
| Summary precedes catalog refresh; accumulation uses bounded/current-run inputs | SRC-1; RT-RTE-append-ingest, MEM-OBJ-source-pages, MEM-OBJ-synthesis | Conditional continuation after 2000 raw lines, current-run mention counts, five discretionary full summary reads | Complete source incorporation, historical re-review or freshly catalog-grounded thesis production | Defined accumulated-source selection, corrected ordering and inspectable consumed-input/output lineage |
| Query knowledge landing conflicts with authoritative write scope | SRC-1; RT-CLM-query-safety, RT-RTE-query-answer, MEM-OBJ-question-pages | Normal query Q.5 versus conventions; stricter fallback separately supported | One coherent compliant save-back recipe or blanket read-only guarantee | Consistent admission contract and observed writes under that contract |
| Raw preservation is policy, not byte or filesystem immutability | SRC-1; RT-CLM-raw-policy, MEM-OBJ-raw-assets, MEM-OBJ-raw-documents | Ingest-time asset rewriting, builder overwrite recipe, PDF collision deletion and NLM failure sentinel | Universal consent enforcement, untouched original bytes or guaranteed valid acquisition | Consistent re-ingest controls, error-aware admission and preserved acquisition provenance |
| GitHub destination and mutable-cache seams | SRC-1; RT-RTE-github-ingest, MEM-OBJ-repo-specifications | Staged-spec copy to raw despite wiki pointers; owner/repo cache refreshed without locking | Reliable local specification delivery and revision-stable concurrent reads | Correct destination recipe, commit-isolated or locked reads and delivery trace |
| Catalog projection is narrower than documented maintenance | SRC-1; RT-CLM-index-contract, MEM-OBJ-index-metadata, MEM-OBJ-index-md, MEM-OBJ-log | Optional prior-index input, incomplete dedup, absent ingest_log, no YAML count refresh, wiki/notes omitted from enumeration | Complete uniqueness, live canonical counts/history, automatic note navigation or established log-tally consumer | Explicit preservation/validation inputs and implementations for history, counts and omitted navigation |
| Structural checks and semantic flags have limited licenses | SRC-1; EPI-RTE-lint-mechanical, RT-RTE-lint-maintenance | Body-wikilink predicates, stripped anchors, pointerless skips; wiki-only eight-page judge; link-only hub detection | Factual acceptance, heading/citation fidelity, complete missed-promotion detection or automatic withdrawal | Checks for the intended domains and evidence-consuming dispositions with consequential consumers |
| Comparison writer exists without established integrated dispatch | SRC-1; MEM-RTE-comparison-writing, EPI-CLM-comparison-repair | Numbered ingest pipeline and render's four formats; lint recommends unsupported comparison format | Routine comparison production or working literal repair command | Explicit supported dispatch and provenance-linked write/refusal trace |
| Export delivery and render freshness remain qualified | SRC-1; MEM-OBJ-research-appendix, RT-RTE-render-output, MEM-OBJ-chart-script | Documented writer role; prompt/path noop; embedded chart constants | Deployed appendix delivery, always-fresh rerender or source truth from execution | Delivery trigger and consumer evidence, content-sensitive refresh and checked data lineage |
| Upstream provenance, visual form and semantic novelty unresolved | SRC-1; MEM-OBJ-index-summary, MEM-OBJ-key-highlights, MEM-OBJ-raw-assets, EPI-CLM-synthesis-confidence; profile verification | Initial histories and control; visual payloads; unpaired model transformations | Authenticated source warrant, complete lineage/agency/form unions, proved entailment or new-claim synthesis | Origin/admission histories and paired premises, outputs and checked interpretations |
| Theory, learning and self-change loops remain unexercised or incomplete in evidence | SRC-1; RT-RTE-append-ingest, RT-RTE-lint-maintenance, MEM-OBJ-log; profile verification | External-topic thesis and additive criticism; unresolved recorded trace inputs and later log consumer | Observed theory revision/iteration, improved future capacity, own-method reflection, complete exercised autonomy, exercised self-improvement or a complete trace-learning negative | Individuated criticism/blame and changed reliance, retained next-round dependence, capacity comparison, self-representation links and evidence-responsive own-organization update exercised later |