Agent-R synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
Agent-R's strongest supported contribution is an inspectable pipeline for collecting scored agent experience and constructing corrective training material. It is an offline builder/improvement plane, not a captured closed self-training service. This account covers the repository at commit 82fcc1ca7873b460949ed49022146dd988c32e31, with an applicability cutoff of 2026-10-12. SRC-1 supplies implementation and README design evidence. No target execution, generated artifacts, checkpoint comparison or causal experiment was supplied. The boundary is complete artifact, partial loop: collection, revision construction and evaluation are shipped, while the advertised training configuration and checkpoint-promotion machinery are missing. These are source-level capacities and defects, not observations of successful operation. See the boundary and Source register, RT-CLM-iterative-training and RT-ABS-training-machinery.
From a bounded task to retained experience
A caller configures a task server, task range, model directory and search budgets. WebShop and TextCraft selection excludes test indices and tasks outside the clean-training list; SciWorld uses server-provided indices. Local inference loads caller-selected weights from MODEL_DIR. A model-name label does not pin those weights or ensure that a later verifier uses the same checkpoint. The shipped drivers select local inference; the hosted API wrapper is an injectable alternative, not an automatically selected provider. Neither inspected wrapper updates parameters. See RT-CMP-func-call-offline, RT-RTE-tree-collection and RT-RTE-provider-alternate.
The actor proposes text actions, software owns execution and search successors, and the external environment supplies observations, reward and completion. Search generates candidates serially. For each candidate it resets a shared client, replays earlier actions and executes the new action. This reconstructs a branch only if reset and transitions are repeatable and the client is used exclusively. It is not an isolated worker or a saved environment snapshot. Terminal rewards update ancestors; unvisited nonterminal children have priority, otherwise value/exploration rules and random ties select the next branch. Negative rewards become terminal zero-valued disasters. These signals prioritize sampled outcomes; they do not certify the actor's explanation or identify why an action succeeded. See RT-RTE-tree-collection, RT-BAP-model-action and EPI-RTE-reward-check.
SciWorld adds lexical action substitution before new steps, unlike WebShop/TextCraft and replay. Exact matching or positive token overlap can replace the proposal, but a zero-overlap fallback retains it. This is a compatibility heuristic, not an all-actions-valid guarantee. The retained actor response can consequently differ from the executed action. See RT-RTE-sciworld-action and MEM-OBJ-extended-node.
After search returns, collection saves a recursive JSON tree rather than a selected solution. Natural-language interaction content and symbolic score/branch metadata share a file but have different consumers. Reloading reconstructs children and message mappings, not parents, environment clients or a live FastChat conversation. It supports later dataset traversal, not established search resumption. An existing filename suppresses recollection without a freshness or provenance check. See MEM-OBJ-extended-node and MEM-OBJ-node-statistics.
Context shortening is deletion rather than retained semantic summarization. Search trims the actor-call copy and then restores full node history for storage. Ordinary evaluation saves its pruned conversation; the supplied-prefix helper keeps a separate full history. These paths do not establish unconditional context fit: fixed-position deletion can cease making progress, and token counts need not match provider serialization. See the memory annotations and the runtime guarantees.
From retained paths to a synthetic correction example
The representative workflow is WebShop collection followed by a separately invoked revision builder. Both stages and their filesystem handoff are shipped; the caller owns inter-stage scheduling and recovery. The builder pulls trees from a requested directory, selects terminal childless paths and ranks them by average value. This is requested batch consumption of retained experience, not unsolicited memory injection into a later actor. See the workflow account, RT-RTE-revision-export and the memory read-back account.
The shipped export is materially defective. ALPHA and BETA remain environment strings but are compared with numeric values. Every nonempty CLI export path must pass a defective comparison; nonpositive roots or empty candidates can return earlier without delivering examples. Good-only mode and disabling critique do not repair the comparisons. Beneath this defect, pairing compares distinct paths without enforcing the README's same-parent condition. Thus neither successful nonempty export nor adjacent-branch recovery is established by the shipped route. See RT-RTE-revision-export and the report verification.
Conditional on reaching the critique helper, the verifier receives task description and low-path action/observation text, not the successful alternative or reward statistics. It is asked for reasons and a usefulness judgment. Software continues the low prefix when the extracted string contains good or uncertain; otherwise it includes the current node and stops. A hypothetical not good judgment also continues under this substring rule. The judgment therefore has operational authority over prefix length, not correctness-certifying authority or acceptance of a diagnosis as true. See EPI-RTE-critique-check, EPI-RTE-prefix-disposition and RT-BAP-verifier-prefix.
Software then assembles a low prefix, a randomly selected stock error admission, Action: wait, a fabricated ok. acknowledgement and a high-path continuation. It does not replay that assembled sequence to check compatibility with the low branch's reached state. The correction rationale is not authored from the verifier's reasons; those reasons remain sidecar feedback. Copied observations retain their original branch attribution, while the inserted self-error claim is not entailed by them. A constructed example is therefore neither an observed recovery trajectory nor evidence of model self-recognition. Good-only mode omits the synthetic correction; paired no-critique mode uses the complete low path. See EPI-RTE-conversation-splice, MEM-OBJ-revise-log and MEM-OBJ-revise-feedback.
Loss flags express intended training eligibility, but no captured optimizer interprets them. An exposed, separately callable converter copies content into positional conversation fields while dropping roles, loss flags, feedback and an unmatched trailing message. It has no shipped driver or trainer caller. Metadata loss on this afforded branch cannot be attributed to every possible external training path; conversely, JSONL retention cannot establish that reasons or masks reach training. See MEM-OBJ-loss-flags and MEM-RTE-dataset-rewrite.
What the improvement account establishes
Automatic transformation of linked state/action/outcome experience into durable training material is an implemented, conditional trace-fed artifact construction. This is a positive architectural contribution even though nonempty CLI export is blocked. Callable conversion is a separate afforded contribution; parameter training is only declared. These distinctions, checked in profile verification, do not establish improved capacity for future action. No evidence shows that criticism of a consumed theory produced such improvement. The inspected correction route judges actions and constructs examples, not a formulated method theory that is revised and later consumed. See RT-RTE-revision-export, RT-CLM-iterative-training and the epistemic conclusion.
The theory-builder conditions remain separate: 1, a localized formulated theory is not established for this correction route; 2, theory-content-dependent consumption is not established by use of action histories; 3, action usefulness judgments do not establish claim-directed criticism with placement of blame and changed reliance on a theory; 4, retention of logs and feedback does not establish a next round shaped by theory criticism. These are bounded non-establishments, not absences inferred about inaccessible model processing or all auxiliary helpers. Reliable reflective operation is likewise not established by requested critique or stock admissions. Collection does independently implement computational action proposal and successor control after batch launch, but autonomous completion of training and checkpoint admission is not established. See RT-RTE-revision-export, RT-CLM-iterative-training and the report-verification limit.
For the declared collection → export → retraining → later-actor horizon, the improvement objective is better task completion/reward. Scores and judgments shape candidate-example determination at source level, qualified by export defects. The remaining self-improvement links are unestablished: change to the system's own deployed organization, entry of that change into a live behavioral-authority path, and subsequent behavior causally depending on it. Agent-R therefore supports an improvement-directed design, not established exercised self-improvement or demonstrated successful improvement at this boundary. This conclusion is independent of the partial trace-fed construction contribution. See RT-CLM-iterative-training.
The evaluator performs direct sequential rollout, not MCTS or verifier-assisted correction, and writes final reward/history without a shipped later checkpoint-admission consumer. Its referenced test_id/ input directory is missing. Loading a caller-selected MODEL_DIR in a later process does not identify a derivative of this dataset. For the scenario of offline experience collection, source wiring supports the tree-to-builder handoff. For a delivered correction dataset, the numeric defect requires repair. For iterative actor improvement, training ingestion, mask semantics, checkpoint lineage/admission and later use require additional evidence. Replay-checked splices and calibrated critique would change the warrant for correction examples; a traced training/deployment loop would change the architectural assessment; controlled checkpoint comparisons would support benefit attribution only at the treatment level actually varied. See RT-RTE-agent-evaluation, MEM-OBJ-final-result and RT-ABS-training-machinery.
Limitations
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| No supplied execution or causal comparison | SRC-1; RT-RTE-tree-collection, RT-CLM-iterative-training | Frozen collection/export/evaluation source and README; no observed-run or causal-experiment layer | Successful activation, improved future capacity and causal benefit, including a critique-specific effect | Provenance-linked artifacts and execution traces; controlled comparisons with the relevant component independently varied |
| Missing training and promotion machinery | SRC-1; RT-ABS-training-machinery, RT-CLM-iterative-training | Complete tracked tree at the reviewed commit; external Xtuner not captured | Implemented closed improvement loop, actual ingestion/masking, checkpoint adoption and exercised self-improvement | Frozen training configuration/code, input contract, update/checkpoint provenance, admission and later-consumption traces |
| Export thresholds fail on reached numeric/string comparisons | SRC-1; RT-RTE-revision-export | Both good-only and paired CLI construction paths | Successful nonempty export at this revision; bypass modes are not repairs | Corrected numeric parsing in a new boundary and linked export evidence |
| Splice and diagnosis lack correctness warrant | SRC-1; EPI-RTE-critique-check, EPI-RTE-prefix-disposition, EPI-RTE-conversation-splice | Low-path prompt, substring consumer and synthetic construction; no same-parent gate or post-splice replay | Reliable first-error localization, executable recovery transcript and observed self-recognition | Calibrated judgments with validated parsing, branch/state lineage and replay checking of constructed continuations |
| External semantics and initial provenance remain unknown | SRC-1; EPI-RTE-reward-check, RT-CMP-func-call-offline, MEM-OBJ-extended-node | Inspected local wrappers/adapters; environment internals and checkpoints not frozen | Reward correctness, reset/replay equivalence, initialization trust and pinned actor/verifier identity | Frozen server contracts/implementation, transition evidence, initial conversations and exact checkpoint provenance |
| Retention and conversion do not establish downstream use | SRC-1; MEM-RTE-dataset-rewrite, MEM-OBJ-revise-feedback, MEM-OBJ-loss-flags, MEM-OBJ-progress-file, MEM-OBJ-final-result | Complete Python call-site inspection and afforded helper operations | Reasons/masks reaching an optimizer, result-fed learning, progress-driven recovery or complete auxiliary delivery classifications | Actual consumers and input/admission contracts linked to each retained part, including prefix/demo provenance |
| Evaluation activation and task coverage remain incomplete | SRC-1; RT-RTE-agent-evaluation, boundary coverage | Missing test_id/; task JSON contents and method graphics available but uninspected |
Unchanged evaluator runnability, benchmark representativeness and diagram-specific claims | Resolved evaluator inputs, inspected membership/curation evidence and relevant graphics if those claims are required |
| Auxiliary SciWorld epistemic inventory is incomplete; carried report-verification limit | SRC-1; MEM-CMP-context-helpers, epistemic coverage, report-verification Limits | Context helpers inspected; planning prompt requests facts and diagnosis of previous mistakes, but no shipped consumer closes the route; epistemic functions unassessed | Complete comparison or negative about all afforded planning, knowledge and criticism routes; central collection/export findings remain supported | Recorded assessment of these helper functions and their consequential consumers or documented afforded consumer roles |
Source-read log: none. This synthesis uses the supplied accepted members without adding source evidence. No unresolved cross-report conflict was declared; profile verification adds no limit beyond the uncertainties already scoped in its findings.