Reflexion synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
Reflexion implements a useful partial contribution: failed-attempt information can become retained guidance for another attempt on the same task. The strongest supported result is this implemented feedback pathway, not measured improvement. Host code selects tasks, calls models, evaluates outcomes and supplies selected prior material to later calls. Models propose actions, programs and diagnoses; the drivers own continuation and persistence. This is a collection of experiment workflows, not one general-purpose agent runtime. The four families are programming, ALFWorld, HotPotQA and WebShop, and they do not implement equivalent guidance channels (RT-RTE-program-attempt, RT-RTE-alfworld-trial, RT-RTE-hotpotqa-attempt, RT-RTE-webshop-trial).
The evidence is SRC-1 implementation and attributed design at Git revision 218cf0ef1df84b05ce379dd4a8e47f17766733a0, with an applicability cutoff of 2026-10-14. The boundary is a complete parent artifact with partial loops: UCS search and LeetCode integration terminate in unavailable Git-linked dependencies. No target code was executed, and historical logs and serialized results were not examined as run evidence. Consequently the account establishes conditional wiring, not activation, reproduced benchmark results or causal benefit. The analysis concerns the shipped workflows at this revision, not the external paper or later deployments.
From task request to checked candidate
The representative workflow is ordinary HumanEval Python. The shipped launcher selects GPT-4, one pass and two iterations. Its CLI cannot be assumed launchable as frozen: an unconditional import reaches the missing UCS dependency before strategy selection. Conditional on reaching the ordinary strategy, the parent implementation gives a clear request-to-result chain; see the workflow account and RT-RTE-program-attempt.
The operator supplies a benchmark and budgets. Each task supplies a function request, entry point and reference tests. Separate model calls generate at most one retained internal assertion per pass and an initial implementation. Python syntax filtering checks parseability, not the expected answer's truth. An empty internal-test list can leave the initial pass flag true. Generated code then executes under host authority; testing is a judgment after execution, not permission control before effects (EPI-RTE-test-filter, EPI-OBJ-test-feedback, RT-RTE-code-execution).
Internal failure permits a diagnosis call and successor generation if an iteration remains. The successor receives the previous implementation, current feedback, latest hint and original request—not the entire accumulated diagnosis list. Host code admits the hint without checking its explanation. Internal success has a different consequence: the initial candidate receives reference evaluation and the outer loop exits even if that evaluation fails. A revised candidate receives reference evaluation on internal success or the final permitted iteration. Thus reference rejection does not universally cause another evidence-responsive revision (RT-RTE-program-attempt, MEM-OBJ-reflections, MEM-OBJ-test-feedback).
Acceptance here means a task's benchmark-success label under supplied reference tests, not all-input correctness or a validated diagnosis. Delivery is JSONL append, including the last program even when unsolved. Restart counts existing rows and skips that dataset prefix; it does not restore an interrupted task's diagnostic context. Separate Python/Rust validators can later execute saved solved programs and print discrepancies, but do not correct archived labels or trigger repair (MEM-OBJ-program-results, MEM-RTE-result-validation).
Language and strategy choices change this account. Rust generation omits a required helper argument, independently blocking complete generation-to-execution progression; its callable executor and saved-program validator remain separate supported machinery. Immediate-reflexion diagnoses with executed feedback omitted. Immediate-refinement uses feedback with a constant instead of a diagnosis. Visible UCS callbacks assign costs from test failures, but missing search internals prevent a full scheduling account; evaluated best.code can differ from the archived candidate. The intended standalone LeetCode audit is only an afforded scaffold: contradictory argument handling blocks delivery with assertions enabled even if dependencies return, and no output write is implemented (RT-RTE-program-variants, MEM-OBJ-state, MEM-RTE-leet-evaluation).
What survives an attempt, and where it goes
ALFWorld writes failed-task plans to indexed configuration lists and JSON snapshots. When memory is enabled, failure and not-skip predicates admit a generated plan without a semantic quality gate. Later action and reflection calls receive at most the last three plans for that indexed task. Explicit resume restores plans and success flags, conditional on stable environment enumeration. The trial marks success on done, although it separately reads won; goal attainment therefore depends on the external environment contract, not merely the variable's label (MEM-OBJ-alfworld-memory, RT-RTE-alfworld-trial).
HotPotQA notebooks keep one agent per question and retry incorrect agents over five rounds. Depending on strategy, the next call receives the last attempt, accumulated diagnoses, or the latest trace plus latest diagnosis. Main ReAct deletes large observation lines from a copy toward a 1600-token target and halts active attempts at its prompt/step limits; it does not produce a semantic summary. Truncation can fail when removable observations are exhausted, and accumulated diagnoses are not covered by that trace budget. Answer acceptance is normalized exact match to a supplied key, not unrestricted factual truth. Joblib saving establishes a checkpoint writer but no inspected saved-agent reload invocation (MEM-OBJ-last-attempt, MEM-OBJ-hotpotqa-reflections, MEM-OBJ-hotpotqa-checkpoints, EPI-RTE-answer-normalize, RT-RTE-hotpotqa-attempt).
WebShop supplies the decisive contrast. Failed-trial plans persist and reach later reflection calls and rendered histories. The action call instead receives a separate static base prompt plus a character-limited suffix of current interaction, without those plans. Reloading the configuration does not repair this omission. Full terminal server reward determines success, but the server's grading semantics are outside the frozen source. WebShop therefore supports a retained reflection-to-reflection chain, not action guidance through the inspected plan channel (MEM-OBJ-webshop-memory, RT-RTE-webshop-trial).
Across these routes, files and process state retain experience. Plans are interpreted advice at model consumers; success flags and result counts have fixed routing and suppression effects; saved code acquires executable force at a code consumer. Automatic retry supply differs from explicit configuration/file loading. Selection is principally task correspondence, recency, strategy and outcome, rather than demonstrated semantic relevance. Automatic trace-fed guidance writes with later consumers are supported—including WebShop's later reflection consumer—but this architectural trace-learning finding does not establish improved capacity (RT-BAP-plan-guidance, MEM-OBJ-program-results, MEM-OBJ-alfworld-memory, MEM-OBJ-webshop-memory, profile verification).
What the feedback loop establishes
The loops distinguish checking an outcome from validating an explanation. Programming tests, HotPotQA key matching, ALFWorld completion and WebShop reward can govern labels and continuation. They do not independently assess the causal claims a diagnosis might contain. Retrieved HotPotQA paragraphs have unknown source warrant. Actual diagnosis instances were not inspected, so their relation to inputs remains indeterminate: neither checked entailment nor ampliative conjecture is established merely by requesting an explanation. No diagnosis-specific evidence-consuming acceptance and subsequent integration of an accepted explanation is established in these ordinary loops (epistemic bounded conclusion, EPI-OBJ-qa-content, MEM-OBJ-reflections).
The runtime's independent theory and improvement assessments remain separate:
- Theory-builder condition 1, localized content: prompts request whole-unit explanations or plans; actual formulated explanatory content remains uninspected.
- Condition 2, consumption: delivery affords semantic use, but decisions depending on what a theory says remain uninspected. WebShop lacks the action-side plan delivery established in ALFWorld.
- Condition 3, content-directed criticism: diagnostic instructions request mistake-directed criticism; exercised claim challenge and blame placement remain uninspected.
- Condition 4, iteration: retained-input delivery is wired in the named Python, ALFWorld and HotPotQA pathways, and reflection-to-reflection delivery in WebShop. This does not establish that a criticism's content shaped the next round; full UCS iteration is additionally unresolved.
Whole hints/plans are inspectable and replaceable, but fine-grained editing of their assumptions is not demonstrated. Horizons are principally same-task rounds, with ALFWorld resume; result retention alone does not establish cross-problem use. Criticism of a consumed theory improving future capacity is unestablished. Reflective feedback architecture is supported where prior behavior can update retained representations and those representations reach later behavioral input; exercised bidirectional causal dependence remains uninspected. Computational proposal, admission and scheduling are supported, but autonomous theory building remains unresolved because the content conditions are unresolved. Criticism of the workflows' own method texts is not established, so reflective theory-builder status does not follow (RT-RTE-program-attempt, RT-RTE-program-variants, RT-RTE-alfworld-trial, RT-RTE-hotpotqa-attempt, RT-RTE-webshop-trial).
For self-improvement, the strongest supported claim is an improvement-directed retained-input pathway in ALFWorld and HotPotQA over later attempts on the same task. It is dispositional, not exercised or successful self-improvement: later operation causally depending on the change is unestablished. The programming product-revision loop does not by itself show improvement of the system's own organization. WebShop's retained-plan route does not establish action-oriented self-change. Generation clients do not update weights in the inspected calls, but opaque provider internals prevent a broader negative about parameter change or accumulated state (RT-RTE-alfworld-trial, RT-RTE-hotpotqa-attempt, RT-RTE-program-attempt, RT-RTE-webshop-trial, RT-CMP-gpt4).
For repeated attempts on fixed tasks, the architecture therefore supports differentiated experience reuse, not a single universal learning guarantee. For secure execution, durable correctness correction, cross-task learning or accepted explanatory knowledge, the inspected routes do not establish the corresponding guarantees. Python timeouts leave worker threads alive, generated definitions can execute outside that timeout, and loop counters require external calls to return. Temporary files and budgets are not isolation or rollback (RT-RTE-code-execution, RT-BAP-executable-code).
Concrete changes would alter specific conclusions: wiring WebShop plans into the actual action call would establish that missing delivery edge; repairing Rust's helper call and CLI/dependency failures would change reachability; a durable consumer of validator rejection could change correction authority. Traces linking failed evidence, a retained update and content-dependent later action would address activation and exercised self-change. Improvement needs outcome evidence, and attributing benefit specifically to diagnosis needs independent variation of that component rather than a bundled retry comparison. These are distinct evidence requirements, not interchangeable fixes.
Limitations
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| Operation and benefit were not examined | SRC-1; RT-RTE-program-attempt, RT-RTE-alfworld-trial, RT-RTE-hotpotqa-attempt, RT-RTE-webshop-trial | Parent implementation and attributed design; no target execution or historical result-body assessment | Activated guidance, improved capacity, exercised reflection/self-improvement, reproduced performance and causal benefit | Provenanced execution traces connecting updates to later behavior, outcome measurements and appropriately controlled comparisons |
| Missing search/grading internals and defective parent entry points | SRC-1; RT-RTE-program-attempt, RT-RTE-program-variants, MEM-OBJ-state, MEM-RTE-leet-evaluation | Visible UCS callbacks and LeetCode wrappers/scaffold; dependency gitlinks unavailable; Rust/helper and dispatch defects inspected | Complete UCS scheduling/retention, remote grading semantics, launchable CLI and equivalent Rust progression; solved UCS label identifying archived selected code | Dependency sources at declared revisions and corrected callers/entry points, followed by provenance-linked execution and archived-candidate checks |
| External evaluator and effect contracts remain outside the source | SRC-1; RT-RTE-code-execution, EPI-RTE-answer-normalize, RT-RTE-alfworld-trial, RT-RTE-webshop-trial | Local executor/normalization/client rules, not authenticated benchmark validity, environment/server internals or deployment isolation | Universal correctness, factual equivalence, goal attainment from done, safe execution, wall-clock termination and general rollback |
Validated evaluator contracts, authoritative task evidence and inspected/enforced deployment containment with execution evidence |
| Diagnosis content and semantic effects are unresolved | SRC-1; MEM-OBJ-reflections, EPI-OBJ-qa-reflections, RT-RTE-alfworld-trial, RT-RTE-webshop-trial | Requested explanations/plans and delivery wiring, without output instances or diagnosis-specific acceptance | Actual explanatory propositions, entailment or novelty, validated causes, content-directed theory criticism and semantic synthesis classification | Retained producer inputs/outputs, claim-level assessment against evidence and traces of consequential consumption |
| Historical lineage, opaque checkpoints and task identity remain unresolved | SRC-1; MEM-OBJ-alfworld-memory, MEM-OBJ-webshop-memory, MEM-OBJ-hotpotqa-checkpoints, RT-RTE-hotpotqa-attempt | Known subsequent writes and resume selectors; initial dataset/example provenance, resumed-content control and serialized internals not fully assessed; no saved-agent reload caller established | Authenticated lineage, complete checkpoint form/authority, demonstrated cross-run agent use and universal task isolation or cross-question learning | Provenanced initial/resumed inputs, decoded checkpoints, inspected reload callers and stable task/session mapping; reassignment/reset traces |
| Provider state and exact model fixity are unresolved | SRC-1; RT-CMP-gpt4, RT-CMP-gpt35, RT-CMP-text-davinci, RT-CMP-star-chat, RT-CMP-code-llama | Generation adapters, mutable aliases and unpinned model loading, not provider internals | Complete accumulated-memory inventory, immutable model identity and either presence or absence of provider adaptation | Immutable model identities and inspectable use-dependent state/write/consumer contracts or traces |
| Offline consumer coverage is incomplete: carried report-verification limit | SRC-1; MEM-RTE-result-validation, EPI-RTE-result-audit, report verification | Selected solution/is_solved console validators; verifier additionally inspected vendored HumanEval task_id/completion audit and augmented-result write scaffold, whose proposed execution is commented and enclosing suite empty |
Repository-complete comparisons of offline evaluation, persistence, execution/recovery and result consumers; usable vendored execution. Named validators remain console-only | Complete supported records for the separate scaffold, repaired usable execution and inspected result consumers; execution evidence for operational claims |
| Candidate lineage inventory omits a separate authorship unit: carried profile-verification limit | SRC-1; RT-OBJ-cur-func-impl, RT-RTE-program-attempt, profile verification | Profile covers candidate-related feedback, imports and packaging, but not original model authorship of each retained candidate as a separate unit | Complete candidate-by-candidate lineage comparisons; emitted partial units are not an exhaustive derivation inventory | Explicit candidate-generation lineage units supported by accepted records; the existing partial axis and separate authored existence witness remain valid |
Source-read log: none. This synthesis uses the supplied accepted members and carries both verification limits; it adds no source evidence.