SkillRL boundary

Type: agentic-system-analyses/types/agentic-system-boundary.md

Boundary and evidence

SkillRL is in scope as a builder or improvement plane: software that generates reusable textual skills, conditions agent decisions on them, and changes both the skill bank and model policy during training. It also ships the agent-environment rollout machinery used by that improvement process. The intended use of this analysis is to characterize those shipped responsibilities, not to treat SkillRL as merely a skill-retrieval library or as a general-purpose deployed assistant. SRC-1, README.md, verl/trainer/main_ppo.py, agent_system/multi_turn_rollout/rollout_loop.py and verl/trainer/ppo/ray_trainer.py establish this classification.

The declared intent is:

SkillRL is a framework that enables LLM agents to learn high-level, reusable behavioral patterns from past experiences. While traditional memory-based methods store redundant and noisy raw trajectories, SKILLRL abstracts these into a hierarchical skill library. --- README.md

The whole-system boundary covers SkillRL's functional loop across the repository: offline experience processing and skill generation; shipped bank artifacts and prompts; loading, selection and prompt consumption; model-driven environment interaction; training and validation; failure-driven skill admission and persistence; and the policy-training/checkpoint infrastructure those paths use. The repository's shipped training drivers and environment managers are included, not assigned to an imaginary external host. General-purpose inherited examples and independent benchmark baseline trainers are outside the functional target unless a SkillRL path calls them. This is whole-system coverage of SkillRL, not a claim to have audited every feature of its bundled verl training framework.

Entry points and state-loop coverage

The inspected ALFWorld driver invokes verl.trainer.main_ppo with a bank path and dynamic updates enabled (SRC-1, examples/grpo_trainer/run_alfworld_skills.sh). Scoped searches also found the WebShop and Search skill drivers invoking that same entry point with their respective environment and bank settings (SRC-1, examples/grpo_trainer/run_webshop_skills.sh, examples/grpo_trainer/run_search_skills.sh). TaskRunner.run constructs training and validation environments, workers, a trajectory collector and the trainer, then initializes workers and calls fit (SRC-1, verl/trainer/main_ppo.py). These callers are part of the target.

    from agent_system.environments import make_envs
    envs, val_envs = make_envs(config)

--- verl/trainer/main_ppo.py

    trainer.init_workers()
    trainer.fit()

--- verl/trainer/main_ppo.py

The source trace reaches production and later consumption of state, rather than stopping at an API. The offline generator reads memory records and writes a skill JSON; the SFT pipeline parses supplied rollout dumps, generates memories, aggregates a bank, distills reasoning with that bank and emits training pairs (SRC-1, skill_generation/alfworld.py, examples/sft_data_generation/run_alfworld.sh). The SFT documentation explicitly places its initial rollout dump outside that particular pipeline:

The rollout step is not included — run your own model on the target environment, dump trajectories to disk, then run the pipeline here. --- examples/sft_data_generation/README.md

SkillsOnlyMemory reloads a configured JSON bank. The ALFWorld environment manager retrieves skills on reset and formats retained selections into later observations; its shipped prompt contains a retrieved-experience slot. The rollout collector calls policy generation and feeds decoded actions back into the environment (SRC-1, agent_system/memory/skills_only_memory.py, agent_system/environments/env_manager.py, agent_system/environments/prompts/alfworld.py, agent_system/multi_turn_rollout/rollout_loop.py).

    with open(skills_json_path, 'r') as f:
        self.skills = json.load(f)

--- agent_system/memory/skills_only_memory.py

        next_obs, rewards, dones, infos = envs.step(text_actions)

--- agent_system/multi_turn_rollout/rollout_loop.py

The trainer's validation and training update branches call SkillUpdater, admit returned skills to training memory and save step-labelled banks. Those paths and their validation input collection remain inside the boundary. Inspection of the validation branch shows its write target is the training bank, not the validation bank; this is an implementation boundary, not evidence that validation cannot influence training (SRC-1, verl/trainer/ppo/ray_trainer.py).

        self.envs.retrieval_memory.add_skills(new_skills, category='general')

--- verl/trainer/ppo/ray_trainer.py

        save_path = os.path.join(save_dir, f'updated_skills_step{self.global_steps}.json')
        self.envs.retrieval_memory.save_skills(save_path)

--- verl/trainer/ppo/ray_trainer.py

Repository coverage

The tracked tree was enumerated at the frozen revision. “Included” means part of the functional analysis boundary; the inspection column distinguishes source traced now from available material still to be inspected by later jobs. Uninspected files are not access gaps. All rows refer to SRC-1; paths are commit-relative.

Frozen evidence and dependencies

The applicability cutoff is 2026-10-14. The sole frozen evidence identity is https://github.com/aiming-lab/SkillRL at commit 4d984d455b9b02f0bad283cbe9ab3d66a7e5b89d; checkout HEAD matched that pin during inspection. No later branch state, linked paper, external repository, live model listing or downloaded dataset supplies evidence. The overall tier is code-grounded, with documentation separately treated as declared intent or reported claims.

External dependencies include model weights, generation backends, Ray/PyTorch infrastructure, embedding models for embedding retrieval, environment data/services, and OpenAI/Azure model services. The updater's actual model invocation is inspected in SRC-1, agent_system/memory/skill_updater.py:

        response = self.client.chat.completions.create(
            model=self.model,
            messages=[{"role": "user", "content": prompt}],
            max_completion_tokens=self.max_completion_tokens,
        )

--- agent_system/memory/skill_updater.py

The root documentation assigns the SFT training stage to LLaMA-Factory, while the included SFT-data pipeline emits its inputs (SRC-1, README.md, examples/sft_data_generation/README.md). That external trainer's implementation is not frozen here; this prevents conclusions about its actual fitting behavior or reproducibility. Externally supplied rollout dumps likewise prevent reconstructing the provenance of every initial experience from this source set alone. Neither limit excludes SkillRL's shipped RL collector or its offline processors.

No target code, tests, examples, services or model calls were executed. There are no supplied execution traces or controlled comparisons. Consequently this boundary supports source-level operation and attributed design, not observed end-to-end success, performance, quality of generated skills, causal gains or external-service availability. All tracked source paths remain available within SRC-1 for subsequent inspection; no material shipped path is known to be inaccessible.

Source register