AutoSci synthesis

Type: agentic-system-analyses/types/agentic-system-synthesis.md

Bounded synthesis

From sources to retained research context

AutoSci supplies research procedures for an external coding agent and Python helpers for selected deterministic operations. It is not the enclosing agent runtime. This account covers the shipped system at Git revision f5c2fe009c6683b6aa5b668de759a0610c938aaf, with an applicability cutoff of 2026-10-15. SRC-1 supplies implementation and doctrine/design evidence, not an observed research episode or causal experiment. The boundary includes active skills and setup, wiki tools, browser writers, scheduled intake and optional Hiera execution. Other branches, provider internals, deployed host permissions and advertised paper results are not established by this review. See the boundary and evidence and Source register.

The principal progression is literature intake, retained wiki construction, later question answering or ideation, experiment planning and evaluation, then possible paper production. A user invokes /ingest with a paper. The agent interprets sources, chooses entities and merges, and writes Markdown pages; helpers admit typed graph edges and build derived navigation and context. Pages mix natural-language scientific content with symbolically consumed identifiers, statuses and links. Ingest does not require per-page human acceptance. Source interpretation and retention do not establish source truth or accepted new science. Foundations seeding is a separate admission route: it can use Wikipedia or visibly marked model fallback and skips existing foundation pages. See RT-RTE-knowledge-cycle, MEM-OBJ-wiki-pages and MEM-RTE-foundation-seeding.

Later /ask requests load a brief, gaps and index, then instruct the agent to choose at most 15 relevant full pages and synthesize a cited answer. Persisting that answer as wiki content requires a crystallization flag or confirmation; basic answering still permits audit writes and missing-context repair. The compiler and the model have different selection responsibilities. Code extracts and ranks summaries under purpose-specific section budgets and a default global 8,000-character bound; model relevance judgment chooses full pages under an instruction-level page cap. Neither bound guarantees token fit. Each compilation replaces the same brief, regardless of purpose. An ordinary general brief omits experiment and stale-entity sections because both have zero allocation; other purposes can include them. Originals remain reachable, but the brief can omit qualifications and evidence text. See MEM-RTE-context-compilation, RT-OBJ-context-brief and RT-BAP-retained-context.

This context is not a single interchangeable graph. Semantic edges carry asserted relations and evidence; bibliographic citations record citation lineage, not endorsement; browser graph views also include live frontmatter projections that compiler connectivity does not use. First-occurrence semantic-edge deduplication can discard later evidence without judging its merit. These differences matter when interpreting a displayed relationship or a highly ranked summary. See MEM-OBJ-edges, MEM-OBJ-citations and EPI-RTE-edge-dedup.

Initialization, alternate writers and recovery

The representative multi-agent workflow is /init through parallel /ingest and parent fan-in. The parent prepares a source manifest and intended common base commit; workers receive one relative source path and INIT MODE restrictions in separate worktrees. Workers are instructed to withhold incomplete commits. The parent merges successful committed branches in manifest order, handles semantic conflicts, rebuilds shared aids and runs deterministic lint repair. Known paper failures can be checkpointed and skipped; an empty success branch must instead stop for repair. Stash restoration failures retain recovery metadata. This is intended partial delivery with explicit failures, not whole-request atomicity. See RT-RTE-parallel-ingest and the workflow account.

The shipped staging recipe does not establish that this intended payload flow works. Generated raw sources, wiki entities/graph and checkpoints are ignored, while parent and worker staging is unforced. A relative pathname does not copy a source into a worktree; a nonempty commit may contain tracked index/log changes without the new paper payload. Rebuild and lint cannot recover files never transported. Successful Git fan-in therefore establishes transport only for content actually committed, not complete initialized-wiki delivery. Task framing and conditional committed-artifact persistence remain supported; neither an observed failed run nor universal failure is asserted. The report verification preserves this source dependency failure separately from unobserved agent compliance.

Scheduled intake provides another admission path: CI can select papers and invoke ingest without per-paper approval. Its compatible-model alternative is inform-only. CI attempts commit/push, but unforced staging against the same ignore rules leaves new payload retention conditional. Intake also automatically supplies selected recent wiki previews and audit-tail context to recommendation; this coarse automatic supply differs from requested /ask retrieval. See RT-RTE-daily-intake and its memory annotation.

Browser PATCH and direct metadata CLI calls can change retained fields without a skill identity or experiment verdict. Writer ownership is explicitly policy, not enforced authorization. Helper shape checks, atomic single-file replacement and selected linked-worktree rebuild guards have narrower guarantees than the full workflow. Deterministic lint repairs reverse links and defaults, not scientific truth; the initialization lint invocation uses the initial issue count for its exit status rather than certifying a clean repaired state. Reset is scoped deletion, leaving possible projected edges, root manifests and sidecars; raw-source deletion does not automatically retract derived claims. See RT-RTE-browser-write, RT-CLM-writer-contract, MEM-RTE-memory-maintenance and the workflow account.

From experiments to changed reliance

Research orchestration is primarily model-interpreted. /research coordinates ideation, design, execution, later collection, evaluation and optional writing, with progress Markdown supporting later sessions. Interactive gates can stop idea selection or continuation; auto mode delegates those choices. The inspected stage body specifies ending the session and manual collection, so automatic monitoring/resumption is not uniformly established. A sanity run precedes the normal deployment approval gate; even the instructions do not support “no execution before approval.” See RT-RTE-research-verdict.

The strongest scientific-acceptance design separates three functions. First, a model reviewer and foreground agent assess a localized hypothesis against experiment and sibling results, including methodological criticism. Second, a verdict licenses changed reliance, with confirmation unless auto: supported results may validate an idea only in the main-experiment, single-hypothesis case; partial and inconclusive outcomes do not advance its lifecycle; rejection requests a concrete failure reason. Third, supported claims are connected to evidence edges and context, while other branches retain rejection or inconclusive evidence without thereby accepting it. These are specified procedures, not supplied accepted-claim instances. Reviewer agreement is not calibrated certainty, and the foreground reads the reviewer verdict before forming its own. A stored status cannot authenticate this history because alternate writers can bypass it. See EPI-RTE-verdict-check, EPI-RTE-verdict-disposition, EPI-RTE-verdict-integrate and EPI-CLM-verdict-certainty.

AutoSci's strongest supported feedback contribution is retained negative guidance: experimental results can become failure reasons, persist in idea pages, enter bounded summaries and guide later ideation away from the same failure. Filter eliminations, pilot failures and formal rejections retain different evidential force; they are not interchangeable refutations. Automatic outcome-to-guidance updates and later consumption are afforded, while summarization is implemented. Full verdict reasoning is not uniformly persisted or delivered. Whether criticism actually improved future action remains uninspected. See MEM-RTE-negative-memory and RT-BAP-retained-context.

For scientific theory building, the runtime assesses the four conditions independently: localized hypothesis content is afforded; content-dependent consumption, content-directed criticism with changed reliance, and retention shaping another round are each claimed. None is an observed theory-building episode. Page/section addressability is supported at the procedure level, not formal dependency tracking. Improved capacity for future action is uninspected independently of those conditions. Reflection on AutoSci's own method is not established by criticism of an external scientific hypothesis. Full autonomous theory building remains uninspected; automatic selections and gate bypasses establish narrower delegated control, not computation performing every internal role. Exercised self-improvement is likewise unestablished at this boundary and across later research sessions: retained guidance affords a future behavior path, but no subsequent operation demonstrates causal dependence on the update or improvement. These are separate unresolved properties, not a single negative grade. See RT-RTE-research-verdict and EPI-RTE-verdict-check.

Hiera adds executable candidate admission, frozen contract/snapshot checks, budget reservations, local or remote evaluation and receipts. Approval is local state available to a caller, not authenticated proof of human review. A successful receipt requires execution/result conditions and a finite score, not scientific confirmation. Later ledger replay uses scores, bout history and reservations to rank work, avoid duplicate spending and constrain continuation. loop re-evaluates selected existing candidates; tune separately creates immutable configuration children. This is wired outcome-sensitive execution, with no observed performance gain or content-directed theory criticism implied. See RT-RTE-hiera-search, MEM-RTE-ledger-replay and EPI-RTE-hiera-select.

The object called experience is a computed count dictionary; inspected reporting callers expose its timestamp-bearing revision rather than a durable full snapshot driving scheduling. The operative feedback path is the ledger. Finalization reports wiki_writeback: none, and formal confirmation must return to the normal workflow. Digests and a working directory are not an OS sandbox for arbitrary candidate code. See MEM-OBJ-experience, EPI-RTE-hiera-summary and RT-CLM-sidecar-confirmation.

For a question-answering scenario, the evidence supports persistent, bounded context preparation plus instructed source-page selection, not demonstrated answer accuracy. For initialization, it supports a coordinated protocol with a known transport dependency failure. For iterative research, it supports explicit evidence-sensitive verdict and failure-reuse procedures, not observed accepted new science or improved capacity. For exploratory search, it supports executable integrity, accounting and feedback-directed continuation, not evaluator validity or secure isolation. Payload-inclusive tracking/transfer with an inspectable delivery trace would change the initialization assessment; linked source-to-verdict-to-later-action traces would establish exercised research and feedback use; independently varied comparisons would be needed to attribute gains to retained guidance or a reviewer component. See RT-RTE-knowledge-cycle, RT-RTE-parallel-ingest, MEM-RTE-negative-memory and RT-RTE-hiera-search.

Limitations

Limitation Affected SRC IDs and record citations Inspected boundary Conclusion prevented Evidence that would resolve it
No observed research or causal comparison SRC-1; RT-RTE-knowledge-cycle, RT-RTE-research-verdict, MEM-RTE-negative-memory Frozen implementation, prompts and scaffold; no registered run/experiment Actual compliance, accepted new science, activation, improved capacity, exercised self-improvement and causal component benefit Provenance-linked episodes through later consumption; objective-specific outcomes and independently varied comparisons
Initialization payload handoff is not established; carried report-verification limit and reconciled disagreement SRC-1; RT-RTE-parallel-ingest, MEM-OBJ-checkpoints, memory annotation, workflow account, verification limit Ignore rules, unforced staging, common-base worktrees and fan-in Automatic source/manifest inheritance, complete generated-output transport, delivered initialized-wiki persistence. Memory describes task framing and committed persistence more broadly; corrected runtime/workflow retain the missing payload conditions Tracking/staging or alternative transfer conditions plus inventories verifying required inputs and returned payloads before cleanup
CI writeback need not retain new payloads; carried report-verification limit SRC-1; RT-RTE-daily-intake, memory annotation, verification limit CI staging/commit/push against shipped ignore rules Unconditional cross-run repository persistence for new paper/graph/raw content; not a claim that tracked changes cannot persist Actual tracking rules, staged-payload inventory and CI trace through push and subsequent reads
General-versus-purpose brief wording remains a reconciled disagreement SRC-1; RT-OBJ-context-brief, RT-RTE-knowledge-cycle, MEM-RTE-context-compilation Runtime's broad compiler inventory versus memory's explicit zero general budgets Inference that every ordinary brief includes experiments or stale entities; verifier dismisses a publication defect without licensing that inference A purpose-qualified common account; existing memory record already supports the narrower budget-specific conclusion
Coarse authority units; carried profile-verification limit SRC-1; MEM-RTE-context-compilation, MEM-OBJ-checkpoints, MEM-RTE-ledger-replay, EPI-RTE-hiera-select, verification limit Partial authority inventory grouping selector and recovery/Hiera controls Counting individual mechanisms or comparing authority coverage per consumer; supported force findings remain valid Accepted inventory split by retained part, actual consumer and effect
Summary loss, freshness and withdrawal are scoped SRC-1; MEM-RTE-context-compilation, MEM-RTE-memory-maintenance, EPI-RTE-edge-dedup Budgeted brief, first-occurrence cleanup, direct edits and scoped reset Full rationale delivery, universally fresh derived context, complete evidence preservation or whole-memory erasure Dependency/withdrawal coverage and traces comparing retained originals, summaries and residual stores
Scientific checks lack instantiated evidence and calibrated warrant SRC-1; EPI-RTE-verdict-check, EPI-RTE-verdict-disposition, EPI-CLM-verdict-certainty Prompted review and conditional disposition; alternate status writers Truth or generalization from status, calibrated certainty from agreement, authenticated acceptance history Raw results and provenance, executed checks against named criteria, independent-review/calibration evidence and disposition history
External runtime and model behavior remain unresolved SRC-1; RT-CMP-claude-code, RT-CMP-review-llm, RT-RTE-review-call Shipped integrations, not provider internals or actual host grants Exact model fixity, provider learning, foreground compaction/isolation, deployed permissions or reliable reviewer independence Pinned provider/runtime identities, relevant host traces and deployed permission/configuration evidence
Hiera accounting is narrower than scientific or security guarantees SRC-1; RT-RTE-hiera-search, MEM-OBJ-experience, EPI-RTE-hiera-select Admission/evaluation/replay/reporting source; no instantiated evaluator or execution Authenticated human approval, scientific confirmation, secure confinement, stable outcome-only experience revision or experience-digest-driven scheduling Trusted instantiated evaluator and run evidence; authenticated approval and confinement controls; explicit stable revision and consumer wiring
Included alternatives are not exhaustively assessed SRC-1; MEM-OBJ-wiki-pages, MEM-OBJ-index, RT-RTE-hiera-search, EPI-CLM-validated-ideas, boundary Partial conversion/regeneration, proposal, publication and translation inspection; opaque payload/provenance alternatives Complete mechanism/evaluator inventory, extraction or dissemination fidelity, translation equivalence, or system-wide absence of additional acceptance/learning routes Accepted records tracing the named remaining transformations, payloads, admission controls and consumers at the same frozen source