Ecdysis synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
Ecdysis implements a harness-improvement plane: it collects scored task experience, requests collaborative diagnosis and repair proposals, delegates candidate editing, and retains a candidate only when its aggregate training score strictly exceeds the current baseline. Its strongest supported contribution is this connected evidence-to-proposal-to-admission-to-later-execution pathway. The unresolved question is whether review content produces better future task performance. The shipped complete example cannot answer it: its editor ignores the proposals, and its rewards depend on a predetermined revision predicate (RT-RTE-candidate-admission, RT-BAP-review-editor).
This is a source-based account of the shipped artifact at cf93866d545b0974dbb0bc83b39c31fbbdeeecb8, with applicability cutoff 2026-10-13. The boundary covers all tracked paths, including both CLIs, public APIs, prompts, synthetic driver, persistence utilities and test consumers. SRC-1 is the frozen repository in its Source register. Evidence comprises inspected implementation, declared design and attributed README results; no target code, tests, examples or provider calls were executed. Production task runtimes, editors, benchmarks, provider internals and the linked paper are outside the captured evidence. This synthesis uses the accepted members without additional source reads.
Collection determines what review can diagnose
The caller supplies the harness, collector, editor and review client. The trainer owns a configured, finite round loop; callbacks own task execution and candidate construction. For finite rewards, failure extraction selects available values below the threshold, clips message tails and groups records by termination reason and low reward components. Missing rewards are excluded. Nonfinite inputs require a separate qualification: NaN can enter review despite not being below threshold, and the default scorer does not validate nonfinite values (MEM-OBJ-trajectories, EPI-RTE-score-check, report verification limit). Source basis: SRC-1, src/ecdysis/training.py and src/ecdysis/evolution.py.
Recurrence across distinct recorded task IDs determines group priority, not eligibility: singleton groups remain auxiliary review input. Models receive all group summaries, up to eight displayed task IDs per group and the first twelve failure metadata samples. They do not receive trajectory messages, task descriptions or current harness code. The editor receives fuller compacted failures, groups, harness and specification. Thus the model can propose explanations from signatures, but cannot inspect omitted actions. Repeated signatures also do not distinguish a systematic harness defect from a recurring task-model limitation (RT-RTE-failure-review, EPI-RTE-failure-shaping). Source basis: SRC-1, src/ecdysis/fdcr/review.py.
Collaborative review produces advice, not a verified explanation
Code schedules Analyst, Critic and Engineer serially for each pass, then Moderator. These are role-conditioned requests through one client, not independently launched agents or separate permission principals. Each fresh request contains a role prompt, reconstructed failure context and the last twelve transcript turns. The ordinary two-pass review makes six text calls and one JSON call. The final Engineer still receives a call although instructed to take no action. Critic is asked to challenge overfitting, false triggers and harmful repairs; subsequent Analyst turns are asked to revise. Moderator proposes the final specification (RT-RTE-failure-review). Source basis: SRC-1, src/ecdysis/fdcr/review.py and src/ecdysis/fdcr/roles.py.
Full replies remain in memory and optionally checkpoint JSON, but only the history tail enters later model requests. There is no on-demand model retrieval of omitted turns or a token budget. The Moderator's request to read the full transcript does not override this selection. Proposal descriptions and rationales can preserve reasons; their presence, actual criticism and subsequent semantic use are unobserved (MEM-OBJ-transcript, EPI-OBJ-proposed-changes).
Moderator normalization retains four specification fields. Unparseable or non-dictionary output can become empty fields without preventing editing; other malformed field conversions can raise. Neither normalization nor a displayed instruction to implement agreed changes verifies agreement or truth. The specification has advisory force at the supplied editor, which owns implementation. No evidence-consuming acceptance of an explanatory claim is established by this handoff (RT-OBJ-review-spec, RT-BAP-review-editor, EPI-RTE-spec-render). Source basis: SRC-1, src/ecdysis/fdcr/review.py and src/ecdysis/training.py.
Score admission changes the retained reference under adapter conditions
The trainer evaluates the candidate and admits it by candidate_score > baseline_score. The default score is the mean of available rewards, not an answer oracle or a check of the proposed explanation. Under fixed, comparable tasks/model/environment and isolated, nonmutating callbacks, this gate selects a higher measured training score for subsequent collection and inference. It does not enforce those external conditions, held-out generalization or semantic compliance. Rejection keeps the previous reference but cannot undo callback mutation. A frozen result dataclass does not deep-freeze its harness (RT-RTE-candidate-admission, RT-OBJ-frozen-harness, EPI-RTE-score-check). Source basis: SRC-1, src/ecdysis/training.py.
The representative workflow is the shipped quickstart, selected because one caller connects collection, review, editing, admission and later delivery. Its editor increments a dictionary revision regardless of review content. Source logic gives revision one a higher reward; the next round records no_failures without review or editing. Equal/lower candidates are rejected, and configured rounds continue after either rejection or no failures. Delivery is a TrainingResult containing the retained harness, final score and decision tuple, followed by later batch task outputs. These are source-supported branches, not observed successes. Source basis: SRC-1, examples/quickstart.py and src/ecdysis/training.py; see MEM-OBJ-round-record.
The installed entry points do not perform this complete workflow. ecdysis prints failure-group counts; ecdysis-run imports user-specified factories and executes their harness directly. Direct inference and benchmark execution bypass training admission. Inference schedules no further adaptation, but external executors still receive live mutable arguments. Batch execution normally records executor exceptions and continues; direct single-task inference propagates them. An exception-free result counts as successful even when its answer is wrong (RT-RTE-harness-inference, EPI-RTE-outcome-check). Source basis: SRC-1, src/ecdysis/__main__.py, src/ecdysis/benchmark.py and src/ecdysis/inference.py.
Persistence preserves several different kinds of authority
Optional FDCR checkpoints retain transcript, progress and completed specification. Reentry checks formatted-context/pass-count hash and model-name equality, then resumes unfinished calls or returns completed advice without new calls. The key omits raw messages, some failure details, role-policy versions and provider endpoint. Matching state is therefore not complete evidence identity or renewed endorsement. Progress controls skipped operations; cached specification remains advice at the editor. Recovery is review-local, not persisted trainer-round restart, mutation rollback or exactly-once provider execution (MEM-OBJ-checkpoint, RT-RTE-checkpoint-recovery). Source basis: SRC-1, src/ecdysis/fdcr/review.py.
Reusable skills are a separate API path. Artifacts save and explicitly reload text plus structural metadata; scope checks and first-ID deduplication establish compatibility and duplicate suppression, not efficacy. Earlier same-ID content can suppress a new tip. Initial provenance, human admission control, rationale retention and downstream agent interpretation remain unresolved. No shipped automatic artifact-to-inference injection exists. Neither inference JSONL nor round audit automatically becomes training feedback (MEM-OBJ-evolved-skill, MEM-OBJ-skill-artifact, MEM-ABS-skill-injection, MEM-OBJ-inference-record). Source basis: SRC-1, src/ecdysis/artifacts.py, src/ecdysis/inference.py and src/ecdysis/evolution.py.
The profile verification supports wired trace-fed retained guidance and adapter-mediated update routes, not demonstrated improved capacity. Automatic history supply is coarse push; explicit checkpoint, artifact and inference requests are pull. Files and in-process stores are established, while opaque harness/payload forms, authority, origins and caller-control alternatives remain unresolved. These local distinctions prevent equating all retained material with learned, validated instructions.
Independent assessments and what could change them
The theory-builder conditions are separately bounded in RT-RTE-failure-review:
- 1 — localized theory content: afforded. Proposal fields permit whole explanatory units; no actual theory instance is established.
- 2 — content-dependent consumption: wired request path. Prior text and content-sensitive revision instructions reach later roles; decisions depending on its meaning remain unobserved.
- 3 — content-directed criticism: claimed. Prompts request challenges and revision, but no transcript establishes challenged claim, result and placement of blame.
- 4 — iteration: wired retention path. Replies reach later turns and optional resume state; retention and use of an actual criticism result remain unobserved.
These statuses do not certify fulfillment of all four conditions. Criticism of a consumed theory improving future capacity is independently unestablished. Adapter-mediated reflection is afforded: failure representations can reach an editor that changes the target harness, but exercised representation-mediated change is unobserved. Reflection on Ecdysis's own review method is not established. Computational scheduling and reference admission are wired without a package-required per-update human gate; an autonomous theory builder remains unestablished because actual criticism and real editor operation are not shown.
Self-improvement is also boundary-dependent. For the shipped Ecdysis improvement machinery, editing external harness products does not establish its own operative self-change. If the assessment includes the supplied task runtime/harness, the records support a conditional, improvement-directed dispositional pathway across rounds and later inference, with aggregate reward as its independently specified objective. Evidence can shape editor input, editing can replace a harness, and that reference reaches later consumers; exercised evidence-responsive change and subsequent behavior causally depending on it remain unestablished. Successful self-improvement is not demonstrated (RT-RTE-candidate-admission, RT-BAP-retained-harness).
For a scenario requiring bounded training orchestration with caller-controlled adapters, the implemented schedule, reference gate and review-local checkpoint path are supported. For a scenario requiring trustworthy causal diagnosis, isolated edits or transfer to new tasks, the evidence does not establish those outcomes. The README's accuracy and speed gains remain attributed claims (RT-CLM-harness-training); this does not refute uninspected experiments. Real adapter code plus a trace linking criticism, patch, admission and later use would establish exercised behavior. Fixed-evaluation held-out comparisons would address future capacity and transfer; independent variation of FDCR would be needed for its component effect. Enforced isolation, comprehensive checkpoint identity and verified skill-consumer wiring would separately change the corresponding architectural assessments.
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| Nonfinite rewards can violate the ordinary failure-selection description; the report verification declares this publication limit. | SRC-1; MEM-OBJ-trajectories, RT-RTE-failure-review, EPI-RTE-failure-shaping, EPI-RTE-score-check; declared limit. | src/ecdysis/evolution.py float conversion and exclusion predicate. |
Universal strict-less-than failure selection and valid-score interpretation for nonfinite input. NaN may enter review. | A finite-input contract enforced before extraction/scoring, or explicit nonfinite handling with corresponding checks. |
| No supplied execution or causal experiment; synthetic editing ignores review content. | SRC-1; RT-RTE-candidate-admission, RT-BAP-review-editor, RT-CLM-harness-training. | Shipped quickstart, tests and README, not real benchmark runs or linked paper. | Observed repair, generalization, cost gains, numerical reproduction or FDCR-attributable improvement. | Authentic adapter/run artifacts and controlled comparisons, with independent FDCR variation for a component claim. |
| Real callback effects and evaluator validity are external. | SRC-1; RT-OBJ-frozen-harness, RT-RTE-candidate-admission, EPI-RTE-score-check. | Generic trainer and inference interfaces; synthetic callbacks only. | Deep isolation/freezing, mutation rollback, comparable real scores or known opaque harness form/force. | Frozen production adapters, evaluation contracts and effect-boundary enforcement or observed evidence. |
| Review omits action evidence and older turns; actual criticism is unobserved. | SRC-1; EPI-RTE-failure-shaping, MEM-OBJ-transcript, RT-RTE-failure-review. | Metadata display and twelve-turn role history in src/ecdysis/fdcr/review.py. |
Full-trajectory diagnosis, causal harness-defect identification, fulfilled theory-building or criticism-mediated capacity gain. | Evidence-bearing transcripts and later dependence on criticism; fuller diagnosis input or an explicit evidence-inspection route. |
| Checkpoint identity and recovery are narrow; rejection does not withdraw cached advice. | SRC-1; MEM-OBJ-checkpoint, RT-RTE-checkpoint-recovery, MEM-OBJ-round-record. | Review JSON/hash/status paths and trainer rejection. | Comprehensive freshness/authentication, end-to-end restart, exactly-once calls or proposal withdrawal on rejection. | Full input/policy/provider identity, authenticated state and coordinated trainer recovery/withdrawal mechanisms. |
| Skills have unresolved origins, reasons and task-agent consumers; outputs are not automatically learning input. | SRC-1; MEM-OBJ-evolved-skill, MEM-OBJ-skill-artifact, MEM-ABS-skill-injection, MEM-OBJ-inference-record. | Shipped artifact construction/load/merge and inference persistence. | Learned skill efficacy, complete ancestry, reason consumption, automatic skill injection or closed outcome-feedback loop. | Provenance-linked artifacts and concrete later agent/converter wiring with consumption evidence. |
| Provider behavior and exact model identity are opaque. | SRC-1; RT-CMP-review-model. | Client requests and model-string checkpoint equality, not provider internals. | Immutable model equivalence, reliable semantic compliance, live provider costs or provider-internal learning conclusions. | Version-pinned provider contract and corresponding invocation/usage evidence. |
The profile verification declares no additional limits. No unresolved reconciliation conflict was supplied; no additional record fault was found in this synthesis.