Auto-claude-code-research-in-sleep synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From research direction to experiment
Auto-claude-code-research-in-sleep, also called ARIS, is a research workflow implemented through host-interpreted Markdown skills and executable helpers. It is not the enclosing agent runtime. This account uses SRC-1, the repository frozen at 26b95cfa0d8747078e9e43b42e20952709e561b8, with applicability cutoff 2026-10-15. The boundary is complete artifact, partial loop: specialist reads extend the initial inspection, but do not establish coverage of every shipped path. Code supplies implementation evidence; skills supply doctrine/design. No target execution, provider call, execution trace or causal experiment supports this account. External hosts own scheduling, permissions and context eviction; external CLIs/providers own model execution. See the boundary and source register.
The representative /research-pipeline connects a user's research direction to idea discovery, an experiment plan, implementation/results, bounded review and a narrative report. It was selected because the shipped parent names the handoff artifacts and completion rules. A foreground model interprets the stages and invokes tools; no encompassing executable controller guarantees compliance. Default AUTO_PROCEED=true announces selection and continues, rather than obtaining human approval. Explicit blocking selection, deployment and review checkpoints can introduce human vetoes. Child skills can also be invoked independently. See RT-RTE-research-pipeline and the workflow account.
The experiment bridge reads the plan and generates scripts. Optional code review may be disabled or silently skipped when Codex MCP is unavailable. Sanity precedes full deployment under the procedure: the child permits two patches plus two clean reimplementations, despite the parent's shorter three-attempt summary. Reimplementation may delete only generated attempt scaffolding, not user source, plans, data or results. Repeated identical rebuild failures or exhausted budget stop with logs. Milestones with at least ten jobs or teacher/student dependencies route to the queue; other cases route to the direct run skill. Queue execution remains uninspected. These are bounded repair and deployment instructions, not observed successful experiments. See WF-RTE-sanity-recovery and RT-RTE-research-pipeline.
Review, claim disposition and delivery
The ordinary Codex MCP bridge launches an external CLI, streams activity, and returns the last agent message plus a thread ID. A reply reloads saved model/configuration/sandbox/working-directory options and asks the CLI to resume its conversation. Missing or malformed option files allow defaults. Model strings do not pin immutable weights. Reviewer inputs form a separate task context, but shared files, persistent threads and streamed events preclude strict frame isolation. Nightmare review bypasses MCP through direct Bash. Neither route establishes deployed sandbox strength or actual reviewer independence. See RT-RTE-codex-call, RT-CMP-claude and RT-CMP-codex.
Review scores do not directly authorize stopping. The deterministic transition helper returns stop/continue/escalate/unavailable from supplied fields; positive review requires score at least six and verdict ready or almost, with backend-specific conditions. Compatibility Copilot positives require a finalizer; native Copilot uses separate evidence machinery; ordinary Codex positives do not independently attest executor identity. The host must call and obey the helper. Full reviewer response and memory append precede gating. Continuing rounds request implemented fixes and new results before re-review, but may defer expensive fixes. Four-round exhaustion returns unresolved issues, not positive acceptance. See RT-RTE-review-transition, MEM-RTE-reviewer-memory and the workflow sequencing annotations.
Scientific support has a separate route. The result-to-claim procedure asks a reviewer to judge an intended claim against measurements, baselines and caveats. Its yes/partial/no disposition permits confirmation and writing, narrowed claims with supplementary experiments, or a pivot. Low confidence requires inconclusiveness; unavailable adjudication must not become supportive downstream content. The implemented precheck establishes only that cited values occur in local files, not that measurements are authentic or that a claim holds. Missing integrity audit is nonblocking, and a failed audit lowers confidence rather than automatically reversing the verdict. No supplied instance establishes a produced accepted ampliative claim. See EPI-RTE-evidence-check, EPI-RTE-result-judgment and EPI-RTE-result-disposition.
Proof checking has different criteria: explicit hypotheses, discharged obligations, declared uniformity, counterexample searches and fresh blind review of severe fixes. Gaps map to unproven, not refuted; unresolved imports retain conditional status. Advisory restatement drift can coexist with PASS. This is model-followed proof assessment, not an inspected formal kernel or a supplied warranted theorem. See EPI-RTE-proof-check and EPI-RTE-proof-disposition.
Normal delivery is NARRATIVE_REPORT.md plus the pipeline report. A report is not fresh claim adjudication, and review termination can retain an unresolved product. Paper writing is off by default; when enabled, missing venue or manual figures can leave it pending. Optional HTML failure does not block narrative delivery. Thus a final filename, clean return or resume COMPLETE does not prove a paper exists or claims are validated. See the workflow delivery annotations.
What survives into later work
Phase JSON separates completion from reported acceptance. Resume selects the first unfinished phase, but the API does not execute a verifier or inspect verdict contents. Explicit skipping and the Codex variant's policy-enabled provisional advancement are material alternatives to accepted progress. Atomic saves and advisory locks protect cooperating API writers, not arbitrary edits; the pipeline may continue without the helper. This is recovery control, not scientific warrant or learning by itself. See RT-OBJ-run-state and RT-RTE-phase-resume.
The optional project wiki retains paper summaries, hypotheses, experiment judgments, claim scope/status and relationship triples in files. Model-followed audits and semantic edits differ from helper persistence: direct callers can supply proof statuses without executing an audit. Accepted empirical integration is also distinct from storing partial or rejected results. Rejudging pages does not establish cascading withdrawal of old edges. See MEM-RTE-paper-capture, MEM-RTE-evaluation-write, MEM-OBJ-claims, MEM-OBJ-edges and EPI-RTE-result-integration.
Later ideation requests a query pack rather than receiving the whole wiki. Fixed code selects ordered sections, filename-sorted paper theses, negative/mixed idea snippets and recent triples under character budgets. It does not implement advertised semantic rankings, clusters or protected retention of every failed idea; earlier sections can displace failures. Claim bodies, full experiment reasoning and edge evidence do not travel in the pack. A scanner-before-Read procedure skips wiki context on scanner failure or findings; clean scanning is not scientific reacceptance. Delivered gaps guide search, papers supply prior-work context, and failed ideas act as a model-interpreted banlist, not an executable veto. See MEM-RTE-pack-read, MEM-CLM-pack-contract and EPI-RTE-query-consumption.
Reviewer continuity instead preserves full responses and optional reviewer-derived suspicions. Hard-mode prompts automatically supply the whole growing memory without truncation; other paths supply file pointers or request thread continuation. Compact recovery selects model-authored candidate/result summaries for later invocations, not demonstrated host eviction. These procedures trade different information losses and context costs; neither measured savings nor successful recall is established. A separate iteration logger computes structural/human pivot signals from caller-reported consecutive zero-finding counts, not from an independent quality judgment. See MEM-RTE-reviewer-memory, MEM-RTE-compact-recovery and MEM-RTE-stall-pivot.
Revising the harness itself
The strongest supported improvement-directed contribution is an explicit route from usage evidence to inspectable skill revisions, with a later host consumer. Optional hooks retain lossy event summaries; meta-optimize requests diagnosis, explanatory diffs and a bottleneck ledger. A human selects patches and separately invokes meta-apply; a fresh cross-family jury may refuse landing. Backups permit restoration. Implemented provenance stamping records caller identities, verdict handles and content hashes, not correctness or authentic reviewer execution. Stamping occurs after editing and is not transactional rollback. The optional Bash blacklist permits Write/Edit and is not a sandbox. See MEM-OBJ-events, RT-RTE-meta-landing and RT-BAP-skill-instructions.
Automatic trace-fed production of later context or guidance is specified for experiment summaries, reviewer-derived memory, outcome lessons and the next-cycle maintenance ledger. This is a supported designed contribution even without measured benefit; raw logging alone is insufficient, and human-selected live skill adoption is a different admission mechanism. The profile verification confirms that these local findings do not resolve the remaining partial inventory. Their underlying routes are MEM-RTE-compact-recovery, MEM-RTE-reviewer-memory, EPI-RTE-result-integration and RT-RTE-meta-landing.
For harness method revision, theory-builder conditions remain separate: 1 localized content — afforded, through visible skill rules/sections; 2 consumption — claimed, through future invocations; 3 content-directed criticism — claimed, through requested evidence-linked diagnosis and revision; 4 iteration — claimed, through retained bottleneck/report inputs to the next cycle. Conditions 2–4 are not witnessed episodes. Research-product retention separately affords localized hypotheses, claims later consumption/iteration, and leaves the complete criticism episode uninspected; it does not establish harness self-change. See RT-RTE-meta-landing and MEM-RTE-evaluation-write.
Criticism improving capacity for future action remains unestablished. Reflection on the harness's own skills and usage outcomes is designed in both directions, but exercised closure is unobserved. Reflective theory building is claimed, not witnessed. Fully computational autonomy is not established on the maintenance route because patch choice and landing deliberately require a human. At the boundary of ARIS skill organization across maintenance cycles, standing improvement-directed self-change is afforded, with fewer failures/interventions as the stated objective. Exercised self-improvement and successful improvement remain unestablished: no trace closes evidence-shaped update, live changed instructions and causally dependent later behavior. These are independent conclusions, not a ladder or a single negative verdict. See RT-RTE-meta-landing.
For a direction-to-narrative scenario, the source supports connected handoffs, finite recovery and explicit non-success returns. For unattended, scientifically warranted completion, it supports requested procedures and local checks, not observed reliability. Source-linked reviewer inputs/outputs, authentic measurements and crash/resume traces would change the operational assessment. A maintenance trace linking diagnosis to a landed diff and later content-dependent behavior would establish more of the self-change pathway; controlled outcome comparisons would still be needed for benefit attribution. Implemented protected failed-idea selection, warrant-preserving pack delivery and withdrawal propagation would change the corresponding memory assessment. None is inferred from labels or acceptance alone.
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No observed operation or causal comparison | SRC-1; RT-RTE-research-pipeline, RT-RTE-meta-landing | Frozen code and instructions; no target execution or supplied runs | Successful unattended research, measured benefit, improved future capacity or exercised beneficial self-change | Source-linked runs through delivery and changed-instruction reuse; independently varied comparisons for causal effects |
| Host/provider contracts remain external | SRC-1; RT-RTE-codex-call, RT-CMP-claude, RT-CMP-codex | Bridge and workflow interfaces, not deployed host/CLI/provider internals | Immutable model identity, actual permissions/isolation, host eviction or complete provider-memory classification | Pinned deployment contracts and inspectable runtime configuration/history |
| Partial-loop coverage | SRC-1; RT-RTE-research-pipeline, MEM-OBJ-events, MEM-OBJ-review-traces; boundary | Queue, native validators, trace writers, global-log consumers, style extraction, broader literature/formalization, paper audits, installers and platform alternatives remain incompletely inspected | Whole-system guarantees, platform parity, exhaustive memory inventory or system-complete epistemic negatives | Producer/check/admission/consumer inspection of those paths and their deployment evidence |
| Fresh-run reviewer-file segmentation unresolved; report-verification limit | SRC-1; MEM-RTE-reviewer-memory; report verification | Run/receipt initialization and append-only reviewer-memory procedure | Guaranteed clean cross-run isolation or intentional cross-run reuse; within-loop replay remains supported | Explicit reset/segmentation/reuse contract plus corresponding consumer trace |
| Checkpoint inventory omits a focused research-contract route; report-verification limit | SRC-1; MEM-OBJ-compact-files, MEM-RTE-compact-recovery; report verification | Verification found idea-discovery Phase 5.6 requests a contract independently of COMPACT; complete recovery consumer uninspected | Exhaustive checkpoint comparison or inference that COMPACT-disabled operation lacks artifact-based recovery | Complete contract production and later active-idea reload account, including experiment/claim consumers |
| Retention and projection do not preserve full warrant or synchronized withdrawal | SRC-1; MEM-OBJ-edges, MEM-RTE-pack-read, EPI-RTE-wiki-retention | Inspected wiki writers, edge dedup and selective projection | Full scientific scope/evidence reaching ideation, reconciled current graph or universally protected failed-idea retention | Warrant-preserving delivery, explicit edge invalidation/rejudgment rules and protected selection with consumer evidence |
| Authentic empirical/proof acceptance instances unavailable | SRC-1; EPI-RTE-result-disposition, EPI-RTE-proof-disposition | Acceptance instructions and local checks, not a completed research instance | Particular accepted ampliative knowledge, a warranted theorem, or general scientific validity | Actual claims, premises/data, reviewer judgments, scope and disposition/integration traces |
| Maintenance and trust controls are path-specific | SRC-1; MEM-RTE-paper-capture, MEM-RTE-evaluation-write, RT-RTE-meta-landing | Direct edits/APIs, optional write scanning, read gate, procedural landing and receipt checks | Uniform enforced admission, scanner completeness, receipt authenticity or transactional rollback | Enforced coverage of alternate writes, evaluated scanner scope, authenticated verdict linkage and transactional recovery evidence |