ExpeL synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
ExpeL implements two experience-fed guidance chains: failed trials can produce retained reflections for another attempt, and selected trials can produce revised rules for later critique and evaluation. These are automatic writes to behavior-shaping text, not parameter updates. Their write-and-consumer wiring is supported; whether criticism improved future capacity remains unobserved. No execution traces, generated rules, checkpoint corpus or controlled comparison were supplied. The strongest supported contribution is therefore an implemented way to retain and reuse revised guidance, not demonstrated learning efficacy (RT-RTE-training-loop, RT-RTE-rule-admission, MEM-OBJ-reflections, MEM-OBJ-rule-items).
The evidence is static implementation and source-native design in https://github.com/LeapLabTHU/ExpeL at commit e41ec9a24823e7b560c561ab191441b56d9bcefc, with an applicability cutoff of 2026-10-14. The whole-system boundary includes the shipped training, extraction and evaluation drivers, prompts, maintenance, persistence and benchmark adapters. External model providers, embedding dependency internals, simulators, live Wikipedia and the WebShop server are not frozen. SRC-1 is the sole registered source; its coverage and exclusions are in the boundary and Source register. This synthesis uses the accepted members without additional source reads.
Gather experience and recover from failure
ExpeL is a memory and context-engineering system with an enclosing benchmark acting loop. It is not a single continuously learning process. Operators separately invoke train.py, insight_extraction.py and eval.py. The foreground driver schedules tasks, Python code controls attempts and context, models propose action or critique text, and adapters execute parsed actions. Provider calls are synchronous invocations, not independently scheduled worker delegation. The README claims autonomous experience gathering, knowledge extraction and improved performance; the records support the machinery for gathering and later context supply, while the stronger knowledge and efficacy claims remain unestablished (EPI-CLM-experience-knowledge, EPI-CLM-learning-efficacy, RT-RTE-training-loop; SRC-1 README.md, agent/expel.py).
The representative workflow is the README's HotpotQA training command with testing=false and resume=false. Disabling testing is consequential: the training configuration otherwise requests caller response text through input(). The actor receives the task, shipped instructions and examples, observations, and retry reflections when present. It proposes Search, Lookup or Finish actions. Finish terminates the attempt; normalized equality with the supplied answer key determines success. A failed attempt can trigger reflection and another attempt while depth remains. Success or depth exhaustion advances the task. The verdict checks the answer, not the reasoning or reflection, and unsuccessful experience is also retained (workflow account, RT-RTE-training-loop, EPI-RTE-answer-check; SRC-1 train.py, configs/train.yaml, envs/hotpotqa/hotpotqa.py).
Other benchmark predicates have different domains. FEVER checks normalized label equality; ALFWorld imports a simulator win judgment; WebShop requires external reward equal to one. These outcomes control retry and trajectory classification, but do not certify every observation, the producing explanation or future transfer. Wikipedia excerpts enter as acquired evidence with unknown warrant, not independently endorsed facts. FEVER's inspected predicate must also be distinguished from its incomplete standard prompt wiring (EPI-RTE-source-acquisition, EPI-RTE-answer-check, EPI-RTE-outcome-disposition, report verification limits; SRC-1 envs/fever/wrappers.py, envs/alfworld/alfworld.py, envs/webshop/webshop.py, prompts/__init__.py).
Working context resets between attempts and tasks. A separate reflection prompt represents the failed trial; long trial lines are reduced to labels and ellipses until the scratchpad target is met. Generated reflection text crosses into the retry, while archived trajectories remain reachable outside the acting window. This preserves selected guidance rather than the full previous frame; the model receives text by value, not dereferenceable links. No independent semantic acceptance gate checks the reflection (MEM-OBJ-reflections, RT-RTE-training-loop; SRC-1 agent/reflect.py, agent/react.py).
Revise retained guidance
Training classifies and saves task-keyed successful and failed trajectories. Extraction reloads these files, supplies successful/failed pairs or groups of successes plus current rules to a critic model, and requests ADD, EDIT, REMOVE or AGREE operations. Software parses and applies eligible proposals without per-rule human approval. Starting a run or consenting to file overwrite is not approval of its individual retained contents (MEM-OBJ-trial-history, RT-RTE-rule-admission; SRC-1 insight_extraction.py, agent/expel.py).
Natural-language rule text and symbolic counters do different work. ADD initializes a count at two; AGREE and EDIT increase it; EDIT replaces text; REMOVE decreases it, more strongly when the pool is considered full. Nonpositive entries disappear and survivors sort by count. A removal request can therefore leave a rule active. These counters govern membership and ordering, not calibrated correctness. Formatting and duplicate/index checks do not enforce generality, the prompt's operation limits or a hard rule-count ceiling (MEM-OBJ-rule-counts, EPI-RTE-syntax-check, EPI-RTE-counter-disposition; SRC-1 agent/expel.py, prompts/templates/human.py).
The critic is asked to generalize beyond finite trials. Actual outputs could contain imperative tips, reshaping, trial-bounded derivations or new general claims; without payloads, their semantic relation is indeterminate. The inspected extraction, parser, updater and cache paths establish operational admission, not a separate evidence-consuming acceptance decision for each rule's truth or transfer. Retention and later use do not establish accepted knowledge or post-acceptance integration. Logs preserve trial inputs and requested operations separately, but acting receives rule text without a per-rule reason graph (EPI-RTE-insight-proposal, RT-RTE-rule-admission, MEM-OBJ-analysis-logs, MEM-OBJ-rule-items; SRC-1 agent/expel.py).
Reuse guidance and demonstrations
Evaluation reloads state and either extracts fold rules or loads copied cached text. It runs ReAct directly, bypassing the training reflection and trajectory-admission loop. Unless no_rules is set, formatted rules reach the acting model as interpretive guidance or references; even FEVER's MANDATORY wording is not a Python action veto. Cache loading and checkpoint restoration bypass fresh critique. Current-pool withdrawal does not remove older snapshots or cache copies (RT-RTE-evaluation-context, RT-BAP-rules-acting, RT-RTE-checkpoint-restore, MEM-OBJ-rule-items; SRC-1 eval.py, agent/expel.py).
Before evaluation acting calls, code automatically rebuilds a FAISS access store from successful histories and shipped examples. Task or recent thought/action/step text supplies the similarity query. Environment filters, optional reranking, task exclusions and token/count budgets further select examples. A matched document identifies a task's history; code then delivers that task's shortest trajectory by character length, not necessarily the trial that supplied the matching key. It replaces old fewshot text in the accumulated prompt. Per-demonstration token limits exclude the task prefix and other prompt content, so they do not guarantee global context sufficiency. no_rules does not disable this demonstration route, and strategy none skips dynamic refresh without necessarily removing shipped examples (MEM-OBJ-retrieval-docs, RT-RTE-evaluation-context, RT-BAP-fewshots-acting; SRC-1 agent/expel.py, agent/base.py).
Fold isolation differs between extraction and retrieval. Drivers subtract evaluation IDs; compare extraction resolves task identity through idx2task. Success grouping instead tests dictionary enumeration positions against those IDs, requiring insertion-order/task-index alignment; resumed extraction reuses saved groups. Retrieval independently uses the full restored success-history pool, excluding the current task rather than the entire evaluation fold. Neither issue proves that leakage occurred, but together they prevent inferring complete held-out evidence isolation from driver subtraction alone (RT-RTE-rule-admission, RT-RTE-evaluation-context; SRC-1 agent/expel.py, insight_extraction.py).
Delivery consists of logs, selected state snapshots and summaries, not an answer object returned from the agent call. Local text and pickle files preserve experience across processes; vectors are rebuilt in memory. Restore trusts pickle provenance and sets attributes outside driver-specific exclusion lists. Separate writes and resume markers afford continuation, not demonstrated atomic recovery, exact replay or semantic rollback. Step, token and reflection limits bound completed interaction only conditionally: external calls must return, and QA search retries can continue indefinitely (RT-RTE-checkpoint-restore, RT-RTE-training-loop, workflow account; SRC-1 utils.py, agent/react.py, envs/hotpotqa/hotpotqa.py).
Assess the improvement pathway separately
At the retained-rule boundary, across extraction and later evaluation, the independent property assessments are:
| Property | Supported assessment and unresolved question |
|---|---|
| Theory-builder condition 1: localized theory content | Afforded: whole rule and reflection units are inspectable and replaceable. Actual theoretical content is uninspected. Rule indices do not separately expose assumptions or scope conditions. |
| Condition 2: decisions depend on that content | Afforded: critic and actor receive retained text. Content-dependent decisions are not observed. |
| Condition 3: content-directed criticism | Claimed: prompts request failure-sensitive criticism and revision. No generated challenged claim, result and blame assignment establishes that criticism occurred. |
| Condition 4: iteration | Wired: revised rules enter another critique round; retained reflections enter retries. This does not establish the other conditions. |
| Learning from criticism | Improvement is claimed; improved future capacity and attribution to criticism remain uninspected. Automatic trace-fed guidance writes are supported independently of that capacity claim. |
| Reflection | Feedback from represented trials to guidance and later acting inputs is wired. Actual representation-mediated behavior is unobserved; criticism and revision of the method prompts themselves is not established. |
| Autonomy | Computational trial, diagnosis, proposal, admission and retry roles afford autonomy within a bounded operator-supplied curriculum. Exercised autonomous theory production and content-directed criticism remain uninspected. |
| Self-improvement | A dispositional improvement-directed self-change pathway is supported for ExpeL including retained instruction state, with better benchmark performance as the objective. Trace evidence enters update determination, revised rules can change its context policy, and later acting can receive them. Subsequent operation exercising and causally depending on that change is unestablished; exercised and successful self-improvement are not demonstrated. |
These assessments follow RT-RTE-rule-admission and RT-RTE-training-loop, not a single system grade. Neither the acting model nor embedder has an inspected parameter-update route; their exact weights are not pinned. Text-mediated adaptation is therefore supported without inferring either fine-tuning or absence of all learning (RT-CMP-gpt-wrapper, RT-CMP-embedder; SRC-1 models/llm.py, memory/__init__.py).
For the selected HotpotQA experience-gathering scenario, the architecture can retain classified experience after success or bounded unsuccessful attempts and supply derived guidance to subsequent invocations. For evaluating generalized insights, the unresolved questions are whether the generated advice is warranted, changes decisions and improves later performance under genuine holdout. Linked trial/critique/rule/acting traces would establish exercised routes; independent rule acceptance or transfer tests would change the warrant assessment. Explicit task-identity filtering in success grouping and a fold-restricted retrieval pool would change isolation guarantees. Controlled comparisons varying reflection, rules or retrieved demonstrations separately would permit attribution at that component level. Reliable reward-based reporting and repaired benchmark prompt dispatch would change the respective reporting and integration assessments. None of these evidence or implementation changes is present in this frozen account.
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No supplied operation or causal comparison | SRC-1; RT-RTE-training-loop, RT-RTE-rule-admission, RT-RTE-evaluation-context | Static shipped drivers and agent implementation | Exercised feedback, successful delivery, changed decisions, improved capacity or component benefit | Provenance-linked execution/checkpoint corpus and controlled comparisons at the claimed attribution level |
| Generated semantic content and rule warrant are unresolved | SRC-1; EPI-RTE-insight-proposal, EPI-RTE-counter-disposition, MEM-OBJ-rule-items | Extraction, parser/updater and cache consumers in agent/expel.py |
Actual novel/general truth-apt claims, formulated criticism, per-rule truth/transfer acceptance or accepted-claim integration | Generated proposals linked to inputs and revisions, plus evidence-consuming acceptance against named truth/transfer criteria |
| Extraction holdout is conditional and retrieval has a broader pool | SRC-1; RT-RTE-rule-admission, RT-RTE-evaluation-context, MEM-OBJ-retrieval-docs | Fold drivers, success grouping, resume and retrieval pool | Unconditional held-out evidence isolation; also no inference that actual leakage occurred | Task-identity and saved-group provenance checks, retrieval traces, or identity-based fold filtering on both routes |
| Reconciliation records a stale peer-wording conflict | SRC-1; EPI-RTE-insight-proposal, RT-RTE-rule-admission, report verification | Pinned records: epistemic calls runtime's wording unconditional, although runtime now carries the same condition | Treating the stale characterization as a current mechanism disagreement; the verifier dismisses it, and the conditional mechanism remains supported | A corrected peer characterization consistent with the pinned runtime; no new mechanism evidence is needed |
| FEVER standard prompt dispatch is incomplete | SRC-1; RT-RTE-training-loop, RT-RTE-rule-admission, RT-RTE-evaluation-context, EPI-RTE-answer-check, report verification limits | prompts/__init__.py, reflection construction and critique builder: reflection inputs and compare instruction are None; success critique key is misspelled |
Uniformly functioning benchmark integrations or extending the selected HotpotQA workflow to successful standard FEVER training/extraction/evaluation | Compatible reflection/critique dispatch and a supported FEVER end-to-end trace; its inspected label predicate remains supported |
| WebShop textual summaries do not certify full-reward success | SRC-1; EPI-RTE-log-summary, MEM-RTE-log-inspection, report verification limits | utils.py score-marker predicate versus envs/webshop/webshop.py reward-equals-one predicate |
Quantitative equivalence between plotted success and benchmark success; actual prevalence and aggregate error are unobserved | Reward-linked log corpus and a reporting predicate checked against numerical full-reward outcomes |
| Per-update-consumer authority inventory omits reflection input | SRC-1; MEM-OBJ-reflections, RT-RTE-training-loop, profile verification limits | Accepted partial authority inventory; failed-trial experience feeds retained reflection generation but is not separately classified there | Complete per-update-consumer authority comparison, not the supported learning value or two-chain finding | Separate classification of retained failed-trial input at the reflection-generation consumer |
| Auxiliary critiques and reflection archives lack established later consumers | SRC-1; MEM-RTE-auxiliary-critique, MEM-OBJ-reflections, EPI-RTE-critique-cleanup, RT-RTE-evaluation-context | Searched shipped drivers/agent/tools; missing auxiliary template keys and unsupported final reflection_similarity dispatch |
Additional working default maintenance chains, direct archive read-back, exhaustive consumer or semantic-curation classification | Compatible shipped callers/templates, actual archive/auxiliary consumer wiring and payloads showing the effect of cleanup |
| Restore, selection and runtime guarantees remain conditional | SRC-1; RT-RTE-checkpoint-restore, MEM-OBJ-retrieval-docs, RT-RTE-training-loop | Local pickle/attribute restore, string-based fewshot replacement, token budgets and external-call retries | Trusted replacement provenance, exact replay, transactional recovery, automatic semantic rollback, universal strategy correctness or wall-clock termination | Provenance and recovery traces, stronger persistence/selection contracts, and bounded external-call behavior |
| External provenance and instance-level source quality are not established | SRC-1; boundary, RT-CMP-gpt-wrapper, RT-CMP-embedder, EPI-RTE-source-acquisition | Frozen adapters and dependency dispatch; external services/weights excluded; individual dataset records and static examples not fully inspected | Exact weight identity, embedding provenance, current compatibility, authentic external outcomes or exhaustive factual quality | Immutable dependency/model identities, captured service evidence and scoped dataset/example quality checks |