AI Research OS Workshop

Type: ../types/agent-memory-system-review.md

AI Research OS is a local, skill-driven research workflow authored as a Claude Code plugin by Paul Iusztin and presented as an AI Engineer World's Fair workshop with Louis-François Bouchard. It turns user-selected or discovered material from Obsidian, Readwise, NotebookLM, GitHub, YouTube, web pages, and PDFs into a per-topic Markdown research directory. The directory preserves source material, builds LLM-written source pages and cross-source wiki pages, and exposes a canonical YAML index for later queries. The reviewed repository is the workshop and plugin implementation, not a hosted retrieval service (README.md, plugin.json).

Repository: https://github.com/iusztinpaul/ai-research-os-workshop

Reviewed commit: dc6660555266239cd36483cf51b5ab39febacf02

Source directory: related-systems/iusztinpaul--ai-research-os-workshop

Core Ideas

A topic directory is the durable memory unit. Each research topic gets index.yaml, a generated index.md, an append-only log.md, immutable source copies under raw/, and a mutable LLM-maintained wiki/. Wiki pages separate source summaries from entities, concepts, comparisons, questions, contradictions, open questions, repository specifications, and derived renders. The contract treats index.yaml as canonical, index.md as a generated human view, and raw files as evidence rather than normal reading material (CONVENTIONS.md).

A research directory is a personal LLM-maintained knowledge base for one topic. It compounds over time: every ingest adds raw sources AND updates synthesis; every saved query becomes a wiki page; every lint pass keeps it consistent. --- plugins/ai-research-os/skills/research/CONVENTIONS.md

The /research skill is a mode router, not one fixed pipeline. It classifies a request as query, append, deep, or init. Query is the cheap default when a matching directory exists. Append ingests only supplied sources. Deep discovery is opt-in, bounded by fixed fast, light, and deep presets, and searches the available personal-source connectors. Init creates the directory and then follows append or deep semantics. This routing prevents an ordinary question from triggering source discovery or rewriting the wiki (research/SKILL.md).

Context efficiency comes from progressive disclosure and context-isolated fan-out. Query mode reads the index summary first, then a per-source wiki page, then user-curated highlights when available, and finally the raw document only when needed. During ingest, one source writer reads each raw source, entity/concept writers consume only the resulting source pages, and overview/synthesis writers start from the index and page metadata, with a cap of five full wiki-page reads. Researcher outputs are capped at fifteen findings per query, source pages and hubs have word budgets, and discovery JSON is merged by scripts so it does not enter the orchestrator context (research/SKILL.md, source_writer.md, wiki_summary_writer.md).

For any source, read in this order and stop when the user's question is answered: --- plugins/ai-research-os/skills/research/SKILL.md

The system bounds local reads rather than the whole memory. index.yaml itself is always loaded and has no partitioning or token ceiling, so the first layer can eventually become the limiting context surface.

Source preservation and explicit synthesis markers are the trust strategy. The conventions require immutable raw material, stable original_path identifiers, source metadata, source-page backlinks, per-claim wikilinks, and a > Synthesis: prefix for LLM judgments. GitHub inputs carry commit SHA metadata and generated repository specifications are meant to cite commit-pinned code. Lint checks detect orphans, missing hubs, missing comparisons, broken links, possible stale claims, contradictions, and open questions. These controls make the corpus inspectable, but they do not verify that every generated paraphrase is faithful to its source or enforce the entire Markdown contract with a schema (CONVENTIONS.md, source_writer.md, research-lint/SKILL.md).

Deterministic scripts stabilize the access layer around LLM-written knowledge. dedup_findings.py keeps one finding per original_path and maps coarse high/medium judgments to fixed scores. build_index_yaml.py normalizes source metadata, preserves earlier sources on append, lets newly ingested records supersede records with the same path, and orders seeds before scored discoveries. build_index_md.py renders the current wiki tree and source catalog deterministically. The index is therefore reproducible for identical inputs even though source summaries and synthesis pages are model-written (dedup_findings.py, build_index_yaml.py, build_index_md.py).

The adoption surface is ordinary local files with optional external connectors. Obsidian is a viewer and source connector, not the storage engine. A user can inspect and version the research directory with normal file and Git tools, and missing Readwise, NotebookLM, Obsidian, or Git CLIs degrade by source instead of aborting every run. The shipped installation is nevertheless Claude Code-specific: it is packaged through .claude-plugin, names Agent and AskUserQuestion orchestration primitives, and resolves scripts through CLAUDE_PLUGIN_ROOT. The Markdown artifacts are harness-portable; the repository does not ship an equivalent Codex plugin manifest (marketplace.json, research/SKILL.md).

Artifact analysis

  • Storage substrate: files repo — Topic memory persists as Markdown, YAML, JSON, images, PDFs, and generated render files in a local directory; the reusable behavior of the system persists as versioned skill instructions, agent prompts, manifests, and Python scripts in the plugin repository. External services and CLIs supply material, but the reviewed system's standing memory does not require a vector database or hosted store.
  • Representational form: natural-language symbolic — Raw documents, source pages, concepts, entities, comparisons, questions, synthesis, contradictions, and prompts are natural-language. YAML/JSON frontmatter, canonical source records, paths, scores, wikilinks, manifests, Python scripts, and Canvas/chart artifacts add symbolic structure. LLMs generate content, but no model weights, adapters, or other retained parametric state are part of the system.
  • Lineage: authored imported other-compiled — Users author questions, source selections, overrides, and optional wiki edits; the plugin authors provide the skills, agents, and scripts; raw/ holds imported or extracted source material; source pages, hubs, comparisons, synthesis, indexes, distillations, and renders are compiled from retained sources or existing wiki pages. A source change or corpus append invalidates the relevant derived pages and indexes. These are source-to-artifact transformations, not extraction from agent execution traces.
  • Behavioral authority: knowledge instruction enforcement routing validation ranking — Raw and wiki content supplies evidence, reference, and advice to agents and humans. Skill and agent Markdown instructs the harness. Mode gates, opt-in discovery, write-scope rules, and immutable-source rules constrain actions. The canonical index, wikilinks, origin metadata, fixed source scores, and page thresholds route and rank attention. Scripts and lint checks validate structural conditions, while generated indexes enforce a narrower symbolic contract.

Raw source layer. Raw Markdown, PDFs, images, and transcripts are file-backed knowledge artifacts. Direct file copies are imported lineage. Markdown extracted from PDF, HTML, or caption data is other-compiled lineage because the helper changes representational form, even though it remains source evidence rather than a cross-source conclusion. The conventions assign raw files evidence authority and forbid normal mutation.

raw/ is immutable. Files land here on ingest and are never edited. Re-ingesting an existing source overwrites only with explicit user consent. --- plugins/ai-research-os/skills/research/CONVENTIONS.md

This immutability is a natural-language system-definition rule rather than filesystem enforcement. Routine append/deep paths deduplicate against index.yaml, but the builder also says partial reruns overwrite existing destination files without prompting, so effective protection depends on correct routing and agent compliance (builder.md).

Per-source and cross-source wiki. Source pages are other-compiled natural-language knowledge artifacts derived from one raw source. Entity and concept pages consolidate claims across at least two source pages. Comparison and synthesis pages can make explicitly marked cross-source judgments that no source states, giving them synthesize semantics while keeping knowledge-artifact authority. Existing pages can be rewritten or enriched on later ingest, with created, aliases, and <!-- KEEP --> user blocks preserved by instruction (source_writer.md, wiki_page_writer.md, wiki_summary_writer.md).

Indexes and ranking metadata. index.yaml is a symbolic routing and ranking artifact compiled from seed/discovery JSON, the prior index, and the wiki tree. index.md is a generated natural-language/symbolic projection for Obsidian. Seeds are forced to score 1.0; researcher high and medium labels become 0.8 and 0.5; all candidates remain in the index. The scores choose ordering rather than excluding content, and the implementation does not learn them from later use (dedup_findings.py, build_index_yaml.py).

Skills, agent prompts, and validators. The plugin's SKILL.md, CONVENTIONS.md, agent prompts, manifest, and Python scripts are authored repo artifacts with system-definition authority. Natural-language instructions decide the mode, source scope, read order, page thresholds, subagent boundaries, and permitted writes. Python scripts deterministically enforce deduplication, index normalization/rendering, PDF and transcript conversion, and four structural lint checks. The broader content contract remains instruction-level: there is no single validator that proves all frontmatter, citations, source faithfulness, idempotency, or write-scope rules (research/SKILL.md, research-lint/SKILL.md).

Derived exports and audit surfaces. research-distill compiles only the sources used by a target content corpus into a portable research.md, retaining metadata, URIs, relevant claims, exact short quotes, nuances, and wiki pointers. research-render compiles selected wiki pages into slide decks, chart code/images, Canvas graphs, or briefs and registers them in the generated navigation. log.md, contradictions, and open questions retain operation summaries and audit findings. They remain knowledge artifacts; none is automatically converted into a stronger instruction or validator (research-distill/SKILL.md, research-render/SKILL.md, lint_judge.md).

Promotion path. The system promotes external material through imported raw source -> per-source summary -> repeated-mention entity/concept hub -> cross-source comparison or thesis -> optional distillation/render. It also promotes a multi-source query answer into a durable knowledge document plus a slim question pointer. This raises abstraction and navigation salience but not behavioral authority: generated knowledge does not become a plugin instruction, schema, or hard gate. The only system-definition artifacts are shipped, authored plugin content.

Comparison with Our System

AI Research OS and Commonplace both use inspectable, versionable files as the primary knowledge medium; distinguish source material from derived synthesis; treat generated indexes as access structures; and make agent context a staged path through summaries, links, and full artifacts. Both also recognize that the agent-facing read path matters as much as storage. AI Research OS packages these choices into a per-topic personal research workspace with broad source connectors and conversational query/render workflows. Commonplace packages them into typed collections for durable methodology, with collection contracts, artifact schemas, deterministic validation, semantic review, and explicit link vocabularies.

The closest structural alignment is AI Research OS's raw/ -> wiki/sources/ -> entity/concept/comparison -> synthesis ladder. It resembles Commonplace's separation of snapshots, source analyses, library notes, and generated navigation. AI Research OS makes this ladder explicit within every topic and supplies a query read order. Commonplace has stronger artifact-specific contracts and validation, while AI Research OS has a more turnkey ingestion surface for personal sources and a richer set of end-user exports.

The strongest divergence is where governance lives. AI Research OS relies heavily on authored skill text and narrow helper scripts. Its source-page and synthesis agents are told to cite every claim, mark synthesis, preserve user blocks, and obey exact layouts, but there is no repository-wide validator for those promises. Commonplace gives schemas and validators more authority and treats semantic promotion as an explicit workflow. The tradeoff is speed: AI Research OS can grow a useful topic wiki from many sources in one orchestrated run; Commonplace spends more effort deciding which claims deserve durable library status.

The second divergence is mutability. AI Research OS declares wiki/ LLM-owned and rewrites hubs, overview, and synthesis as the corpus changes. Commonplace artifacts are individually authored and reviewed, and broad graph work is separated from the write of one note. AI Research OS favors a continuously regenerated current view; Commonplace favors independently inspectable artifacts whose relations and replacement history remain explicit.

The system's multi-agent decomposition is particularly relevant to Commonplace. Raw sources enter one bounded writer context, downstream synthesis consumes compact source pages, and the top-level orchestrator handles paths and summaries rather than content. This is a concrete context-scheduling architecture, not only a file layout. The weakness is cumulative abstraction error: downstream agents usually do not revisit raw evidence, so an error in a source page can propagate into hubs and synthesis until lint or a human catches it.

Borrowable Ideas

Make the read ladder explicit for each high-volume collection. A Commonplace collection with many source artifacts could declare a default order equivalent to index description -> source analysis -> curated derivative -> raw snapshot. Ready for collections whose existing artifact types already provide those layers; otherwise it needs a concrete workload before adding new types.

Use slim question records as pointers, not answer containers. AI Research OS saves a multi-source answer in the appropriate concept, comparison, repo, or notes page and keeps the question file as a short pointer. A Commonplace workshop could use the same shape to preserve decision prompts without duplicating the resulting note. Ready for a workshop experiment, not yet a global type.

Keep fan-out contexts source-bounded and synthesis contexts derivative-bounded. For large ingests or comparative reviews, one worker per raw source followed by workers over compact source analyses reduces orchestrator load and prevents unrelated source material from mixing. Ready for workflows that already have source analyses; it needs explicit checks against abstraction drift before becoming a default.

Preserve the full source URI envelope in portable distillations. A compact export that retains original_path, source-page path, raw path, commit metadata, and exact short quotes is useful when another agent needs a bounded writing context without losing auditability. Ready for experiment in article or workshop production.

Add a conversational checkpoint before broad synthesis. Presenting top sources, near-threshold concept clusters, candidate comparisons, and open questions lets the user correct scope before many derivative pages are written. Ready for expensive multi-source investigations; unnecessary for one-note writes.

Do not borrow an unvalidated generated wiki as a library authority. AI Research OS's mutable wiki is useful working memory, but Commonplace should keep promotion gates before source-derived synthesis becomes a durable note, instruction, or validator. This needs no new feature; it reinforces the workshop-to-library boundary.

Write side

Write agency: manual automatic — Users choose questions, sources, discovery depth, output locations, redirects, and explicit overwrites; they can also preserve edits in <!-- KEEP --> blocks. Once invoked, the skills automatically copy or extract sources, run bounded research and writer agents, update wiki pages, build indexes, append logs/open questions/contradictions, lint structure, and produce distillations or renders.

Curation operations: consolidate dedup evolve synthesize — Per-source pages and top-level overview condense stored material; discovery and append merge records by original_path; existing entity, concept, overview, synthesis, and saved-answer documents are updated in place as new sources or questions arrive; comparison and thesis pages make explicitly marked cross-source judgments that inputs need not state. The system flags possible invalidation and staleness but does not automatically supersede stale knowledge, decay entries, or promote a retained item solely from recurrence or observed success.

Automatic acquisition is the dominant write path: source CLIs and helper scripts import or transform external documents into raw files, then LLM writers compile the wiki. That does not qualify as trace-learning. The workflow explicitly excludes original conversation transcripts from research directories, and log.md stores short operation summaries rather than sessions, tool traces, trajectories, or learned behavioral artifacts (CONVENTIONS.md).

Read-back

Read-back: pull — Retained research reaches a future agent only when a user or agent explicitly invokes /research query mode, asks another skill to load the research directory, or opens and follows the index/wiki files. The plugin does not monitor prompts, session starts, tool calls, or task identifiers and push matching memory into an agent context.

The pull path is structured. Query mode locates a topic directory, loads the canonical index.yaml, selects summaries or wiki pages for the question, and drills into highlights or raw files only when the lighter layers are insufficient. Direct wiki-shaped questions can pull overview, synthesis, comparison, concept, entity, contradiction, or open-question pages. research-distill provides another deliberate pull path by matching a content corpus against indexed sources and emitting only the supporting claims, quotes, and metadata (research/SKILL.md, research-distill/SKILL.md).

Selection within the pull is a mixture of deterministic and agent judgment. The index orders user seeds and coarse relevance scores, supports filters over origin, tags, dates, authors, GitHub paths, and other metadata, and exposes explicit pointers. The querying agent then judges which summaries and pages answer the request. There is no vector store, automatic situation matcher, measured retrieval recall, or built-in test that compares behavior with and without the pulled memory. Effective context precision and authority are therefore not verified from code.

Humans are first-class consumers. index.md, wikilinks, Obsidian views, source metadata, raw files, renders, lint reports, and logs are designed for inspection and redirection. The same readable artifact serves the human and the agent, while index.yaml is the machine-facing routing surface.

Curiosity Pass

The "OS" is primarily an executable specification. Most behavior lives in natural-language skill and subagent instructions. The Python code covers deterministic transformations and structural checks, not orchestration, retrieval, or semantic truth. The implementation is consequently more inspectable than a hosted service but less mechanically constrained than the architecture diagrams suggest.

Deep discovery is personal-corpus discovery, not general web search. Research rounds search Obsidian, Readwise, and every non-empty NotebookLM notebook. Web pages, GitHub repositories, PDFs, and YouTube videos are accepted as user-supplied seeds, but generic web-search rounds are described as a future extension rather than implemented behavior (research/SKILL.md). This narrows the README's broad "deep research" framing in a useful way: the distinctive value is mining a personal second brain, not replacing a web research engine.

The first disclosure layer can grow without bound. Every query reads index.yaml, and every source keeps a multi-field metadata record and summary. Downstream reads are carefully capped, but no index sharding, summary budget, or pruning mechanism bounds this initial load. Progressive disclosure protects raw and wiki bodies, not the catalog itself.

The GitHub builder instructions contain a destination mismatch. They create wiki/repos/<repo>, then tell the builder to copy the staged repository specification into raw/<repo>/, while registering uri_full as wiki/repos/<repo>/ARCHITECTURE.md. The committed example shows the intended wiki/repos/ layout, so a literal builder could produce an unresolved index target even though the example is correct (builder.md, example index.yaml).

cp -a ""/. "/raw//" --- plugins/ai-research-os/skills/research/agents/builder.md

Evidence preservation and downstream abstraction pull in opposite directions. Raw sources and exact metadata support audit, but normal downstream writers operate on source pages and are explicitly forbidden from reading raw material. This is excellent context isolation when source pages are faithful and a compounding error channel when they are not. The lint system checks graph and cross-page symptoms; it does not compare a hub or thesis back to raw evidence.

Question save-back creates durable memory from conversation without trace-learning. The system can preserve the substantive answer and a slim question pointer when the user bookmarks it or the answer cites at least two sources. The input is a deliberate knowledge query, not a mined session trace, and the output remains a knowledge artifact rather than a learned rule or policy.

What to Watch

  • Whether the GitHub builder's raw/<repo>/ copy target is aligned with the indexed wiki/repos/<repo>/ARCHITECTURE.md path; until then, repository ingest depends on an agent noticing and repairing the instruction mismatch.
  • Whether the project adds a deterministic validator for wiki frontmatter, required sections, source backlinks, synthesis markers, and <!-- KEEP --> preservation. That would materially raise trust beyond prompt compliance and lint heuristics.
  • Whether generic web-search rounds move from the documented extension point into the research pipeline. That would change deep mode from personal-corpus exploration plus supplied seeds into open-web discovery.
  • Whether large topic directories gain index partitioning or a bounded catalog view. Without it, the layer meant to protect context becomes an unbounded always-read file.
  • Whether Codex receives native packaging and harness-specific orchestration instructions, rather than relying on the portability of Markdown skills written and installed as a Claude Code plugin.
  • Whether lint gains source-faithfulness checks or sampled raw-to-source-page audits. That would address the main risk created by compiling every later layer from LLM-written source pages.

Relevant Notes: