Ecdysis report verification

Type: agentic-system-analyses/types/agentic-system-verification.md

Verification

The pinned reports support a bounded account of a shipped harness-improvement plane, with one local numeric-input limit below. No material correction is required. The supplied structural check says none; all four correction-answer files are empty. There is no correction argument to accept or decline.

I independently read all tracked source files, including the seven test files, packaging, exports, both CLI implementations and the quickstart. HEAD equals the revision in boundary, and tracked status is clean. The tracked tree agrees with the boundary's coverage table. These checks support the declared whole-system boundary for the shipped artifact, not inclusion of external benchmark installations, production editors, task runtimes or the linked paper. No target code, tests, examples or provider calls were executed. All additional source inspection uses SRC-1 and supplies implementation or doctrine/design evidence only.

Runtime routes and consequential guarantees

I traced collection, scoring, extraction, grouping, context construction, role scheduling, Moderator normalization, editor delivery, candidate collection, reference admission and later inference directly through SRC-1, src/ecdysis/training.py, src/ecdysis/evolution.py, src/ecdysis/fdcr/review.py, src/ecdysis/fdcr/roles.py, src/ecdysis/inference.py and examples/quickstart.py.

  • RT-RTE-failure-review is supported: each nonempty ordinary two-pass review schedules six text calls and one JSON call. Fresh requests contain the role policy, metadata display and twelve-turn history tail. The final Engineer still receives a call. Raw trajectory messages, task payloads and harness code are not in this display. All groups reach review; recurrence changes ordering and labels, not hard admission eligibility. The numeric-input qualification below applies to failure selection.
  • RT-OBJ-review-spec and RT-BAP-review-editor correctly separate normalized proposal content from its advisory delivery and actual implementation by the editor. Non-dictionary or unparseable Moderator output can yield empty fields, without vetoing editing. This is not a guarantee that every malformed dictionary normalizes successfully: field conversions can raise. The submitted wording does not require that stronger guarantee.
  • RT-RTE-candidate-admission, RT-OBJ-frozen-harness and RT-BAP-retained-harness correctly describe strict scalar comparison and live-reference retention. The trainer supplies no copying, sandbox, mutation rollback or held-out evaluation gate. It recollects each round, continues after rejection or no failures, and returns the harness plus decision tuple. Frozen dataclass fields do not freeze the contained payload. The source excerpts in these records support their conditional guarantees, not external isolation or real improvement.
  • RT-RTE-checkpoint-recovery correctly distinguishes interrupted progress from completed reuse. Hash/model-string mismatch raises; matching completed state bypasses calls. The hash covers displayed context and pass count, not omitted evidence, role-policy versions or endpoint identity. Provider effects can repeat after interruption, and no persisted trainer-round recovery exists. The interrupted/completed test consumer supports the designed route, not observed recovery.
  • RT-RTE-harness-inference correctly covers single inference, batch inference and the benchmark path. Dynamic factories and direct runners bypass training admission. Executor exceptions are caught only in the batch path unless fail-fast is selected; iterable, ID and encoding failures are not covered by that catch. The source supports sequential task delivery and result persistence, not answer correctness or unchanged callback-owned contents.
  • RT-RTE-artifact-utilities correctly separates explicit construction/save/load/merge from training. Dictionary checks, optional load scope, mandatory parent merge scope and first-ID deduplication establish structural delivery, not semantic admission or relevance selection.

RT-CMP-review-model is supported by SRC-1, src/ecdysis/llm_client.py and src/ecdysis/fdcr/review.py: requests use a model-name string and caller-configured service; inspected code does not request parameter updates or pin immutable weights. SDK overrides are possible outside FDCR. Retry and usage statements concern completed SDK calls and inspected exception branches, not provider reliability. RT-CLM-harness-training accurately attributes the README's gains without promoting them to observation or causal support.

The theory-condition assessments in RT-RTE-failure-review remain appropriately separated. Localized proposal fields afford candidate content; content-sensitive revision instructions and transcript delivery are wired. Actual formulated criticism, semantic dependence and retained criticism results remain unobserved. Those wiring statuses do not certify fulfilled theory building. The reflection and improvement accounts preserve the adapter boundary and missing exercised causal links. The quickstart's specification-ignoring editor prevents attribution of its score change to review content.

Memory objects and consumer authority

I checked every memory declaration against SRC-1, src/ecdysis/evolution.py, src/ecdysis/fdcr/review.py, src/ecdysis/artifacts.py, src/ecdysis/training.py, src/ecdysis/inference.py, src/ecdysis/benchmark.py and their shipped consumers.

MEM-OBJ-trajectories correctly distinguishes compacted raw evidence from learned guidance and preserves the all-trajectory alternative. MEM-OBJ-transcript correctly separates full retention, selected model delivery and standalone saving. MEM-OBJ-checkpoint separates progress/control metadata, usage accounting and text-bearing outputs; its later consumers are the resumed schedule and completed-result return. These declarations do not confuse storage with semantic activation.

MEM-OBJ-evolved-skill and MEM-OBJ-skill-artifact preserve instruction/envelope distinctions, bypassable dictionary checks, unresolved reasons and provenance, and incomplete immediate-parent lineage. Source merge order confirms that an earlier parent's same-ID skill suppresses a new unit. The artifact test is a genuine shipped reader for structural comparison, not an agent consuming advice. MEM-OBJ-round-record is audit data rather than a subsequent decision input. MEM-OBJ-inference-record is an outcome record, with exception-based success and potentially lossy output encoding, not a feedback store automatically consumed by extraction.

The scoped absence in MEM-ABS-skill-injection is supported. Direct tracing and independent caller searches found no shipped artifact-to-inference injection. The declared scope leaves arbitrary external factories/editors open. All seven memory annotations agree with the runtime referents: different model/editor evidence, bounded checkpoint freshness, reasons in specifications versus transcripts, unknown initial harness form, automatic scalar admission, explicit skill loading, and inference without package-scheduled adaptation. Memory's integration issues preserve genuine external uncertainties without contradicting the wired paths.

Epistemic functions and workflow

I checked every ledger row and declaration, including their transformation classifications and separate evidence/decision functions. SRC-1, src/ecdysis/evolution.py, src/ecdysis/fdcr/review.py, src/ecdysis/artifacts.py, src/ecdysis/inference.py and src/ecdysis/training.py support the following dispositions:

  • EPI-OBJ-failure-context and EPI-RTE-failure-shaping support imported assertions and deterministic reshaping, not causal diagnosis. The message-change deduction is sound: changed compacted message text can leave formatted context and checkpoint hash unchanged. Counts concern supplied record identities, not authenticated independent tasks.
  • EPI-OBJ-proposed-changes correctly identifies a specification part capable of carrying explanations but not required to do so. The review ledger's indeterminate content relation does not infer ampliative conjecture from model generation alone. Critic checking, Moderator disposition, candidate adaptation, score disposition and retention remain separate functions with bounded licenses.
  • EPI-RTE-score-check supports arithmetic over available rewards, conditional on finite inputs for the stated arithmetic license. It explicitly notes missing-reward exclusion and missing nonfinite protection. Score evidence does not warrant a repair explanation, evaluator validity or transfer.
  • EPI-RTE-spec-render supports a display transformation plus imperative scaffolding, not an agreement check or coding-agent execution. EPI-RTE-stability-checks correctly gives separate signed-delta and score-drop predicates, including the decline countercase and lack of a shipped consequential consumer. EPI-RTE-outcome-check correctly restricts success to caught-exception status.
  • EPI-OBJ-review-transcript, EPI-OBJ-round-record and EPI-OBJ-evolved-skill have the identities connected by reconciliation. Their possible truth-apt content does not establish actual accepted propositions. EPI-CLM-learned-behavior correctly contrasts a learned-behavior label with utilities that accept manual content and do not establish efficacy.

The epistemic bounded conclusion is supported: retention and operational policy selection do not themselves constitute evidence-consuming acceptance or lifecycle integration of an explanatory claim. It does not turn missing run evidence into absence of implemented capacity or bundle learning, reflection, autonomy and self-improvement into one negative.

The workflow account follows one actual shipped caller rather than joining hypothetical capabilities. Independent reads of the quickstart, trainer, review and inference implementations, and training tests confirm request ownership, textual role handoffs, editor delegation, strict admission, rejection/no-failure paths and delivery. The four workflow annotations match their runtime routes. Checkpoint recovery is explicitly optional and not enabled by quickstart; no successful execution or content-responsive repair is inferred.

I checked all four declared Part of: relations: proposed changes belong to the specification; failure shaping, score production and exception reporting belong to their respective broader runtime routes. These are material functions or content parts, not candidates replacing their parents. I checked all three reconciliation amendments. Transcript and round-record supersessions connect identical referents. The skill supersession connects the same instruction units while preserving optional provenance findings and the separate artifact envelope. No finding is lost or promoted by these identity connections. Trajectory/display overlap is a transformation-chain overlap rather than complete duplicate identity. Checkpoint, transcript and specification remain valid container/content distinctions. The integration dispositions cover memory's issues and the supplementary epistemic helpers. No unresolved cross-report disagreement, missing consequential part or unsupported supersession was identified.

Selected omission questions came from the frozen tree, exports, installed entry points and producer/consumer dependencies, not only the reports' anchor lists:

  1. Could a shipped caller load skills into later execution or automatically turn inference output into learning input? I followed artifact callers, the benchmark factory path, both result formats, exports and tests. SRC-1, src/ecdysis/artifacts.py, src/ecdysis/inference.py, src/ecdysis/benchmark.py, src/ecdysis/evolution.py, src/ecdysis/__init__.py and tests/test_artifacts.py support the scoped absence and format distinction already retained in the records. External integrations remain possible.
  2. Could another shipped control path impose recurrence, rollback, convergence stopping or stronger admission? I traced both CLIs, training branches and caller searches for the stability helpers. SRC-1, src/ecdysis/__main__.py, src/ecdysis/training.py, src/ecdysis/evolution.py and src/ecdysis/benchmark.py confirm the separate paths and absence of helper consumption in shipped training. No stronger package guarantee was found.
  3. Could persistence preserve complete evidence identity or an end-to-end restart beyond the reported checkpoint route? I traced checkpoint producers, interrupted/completed consumers and per-round naming through SRC-1, src/ecdysis/fdcr/review.py, src/ecdysis/training.py and tests/test_evolution_checkpoint.py. The narrow hash and review-local restart account holds.

Independent searches covered src, examples, tests, README.md and pyproject.toml for artifact loaders/builders/mergers, all-trajectory extraction, convergence/regression, specification rendering, standalone transcript saving, checkpoints and tool/hook dispatch. Full source reads also checked producer/consumer wiring. Optional expansion stopped because these consequential alternatives have supported dispositions, every declared record and relation was checked, and no remaining identified material challenge lacks a disposition. External operation, causal gains and production adapter behavior cannot be compared from this frozen source; the reports already withhold those conclusions.

Blockers

none

Limits

  • Failure-selection descriptions should not be extrapolated to arbitrary nonfinite rewards. MEM-OBJ-trajectories, RT-RTE-failure-review and EPI-RTE-failure-shaping describe below-threshold selection, while EPI-RTE-score-check already records the lack of nonfinite reward validation. SRC-1, src/ecdysis/evolution.py converts rewards to floats and excludes a failure when its reward is missing or greater than/equal to the threshold. A NaN comparison does not satisfy that exclusion, so NaN can enter failure review despite not being below threshold. Withhold a universal strict-less-than guarantee or valid-score interpretation for nonfinite inputs. This local exceptional-input omission does not change the finite-score workflow, metadata visibility, strict candidate-reference comparison, memory consumers or epistemic licensing conclusions.

reward = reward_info.get("reward") return float(reward) if reward is not None else None --- src/ecdysis/evolution.py

if failures_only and (reward is None or reward >= failure_threshold): continue --- src/ecdysis/evolution.py