SkillX boundary

Type: agentic-system-analyses/types/agentic-system-boundary.md

Boundary and evidence

This run analyses how SkillX constructs reusable skill libraries from agent experience, maintains and persists them, and prepares their later use in agent prompts. It covers the complete shipped repository, not only its extraction API. The target class is memory/knowledge/context-engineering system: its retained state comprises planning, functional and atomic skills, and its construction and consumption paths issue model calls. Its construction pipeline also supplies a builder or improvement plane, while its exploration workers supply narrower agent loops. These are included responsibilities, not separately frozen targets (SRC-1, core/skill.py, pipeline.py, inference/skill_usage.py, expansion/env_worker.py).

The source-native intent is explicit:

SkillX is a fully automated framework that constructs a reusable, plug-and-play skill knowledge base for LLM agents from experience. --- README.md

The boundary includes trajectory loading and normalization; model-driven plan and skill extraction; quality and tool-schema filtering; embedding-based clustering and model-driven merging; library updates and JSON persistence; exploration, task synthesis and task-generation checkpointing; retrieval, selection, plan rewriting and prompt formatting; shipped benchmark-facing consumers; schema acquisition scripts; sample libraries; and retained experimental result artifacts. These inclusions follow shipped imports and callers, including the command-line branch in pipeline.py, the library factory in inference/skill_usage.py, and all three agent wrappers in inference/benchmarks/ (SRC-1).

The initial trace reaches retained-state production and later consumption. The pipeline updates a SkillLibrary, saves per-epoch checkpoints and final results; the library implements save/load; the inference factory reloads it and constructs a retrieval service; benchmark wrappers call that service to prepare prompts (SRC-1, pipeline.py, core/skill.py, inference/skill_usage.py, inference/benchmarks/appworld.py, inference/benchmarks/bfcl.py, inference/benchmarks/tau2bench.py). This establishes shipped responsibilities, not successful execution or artifact-format compatibility across every path.

checkpoint_path = checkpoint_dir / f"skill_library_epoch_{epoch}.json" skill_library.save(str(checkpoint_path)) --- pipeline.py

library = SkillLibrary.load(library_path) embedding_service = EmbeddingService(base_url=embedding_url) --- inference/skill_usage.py

Maintenance instructions are part of the target. The extraction prompt asks for modifications, additions or retention of existing skills; the pipeline separately filters, merges and updates library state. These prompt instructions are design evidence, not proof that model outputs obey them (SRC-1, prompts/skill_prompts.py, prompts/registry.py, filtering/pipeline.py, pipeline.py).

You have three options: [modify, add, keep] - modify: revise an existing skill to make it more effective (e.g., improving documents). Only change content when necessary, and ensure the resulting skill remains broadly reusable/general-purpose. - add: introduce a new skill only when existing skills cannot support a critical step, in order to improve future performance. - keep: Preserve the skill unchanged when there are no clear issues. --- prompts/skill_prompts.py

Whole-system means all material shipped responsibilities are within scope. It does not mean the repository supplies a complete benchmark deployment. The inference wrappers prepare prompts but leave task execution unimplemented. The expansion workers do contain model/environment interaction loops and evaluator calls; those shipped loops remain included rather than being mistaken for external callers (SRC-1, inference/benchmarks/appworld.py, inference/benchmarks/bfcl.py, inference/benchmarks/tau2bench.py, expansion/env_worker.py, expansion/env_explorer.py).

async def run(self, task: str) -> Dict: """Run agent on a task.""" raise NotImplementedError("AppWorld execution requires environment integration") --- inference/benchmarks/appworld.py

External dependencies are outside this frozen source: model-provider implementations and live responses; the embedding server; benchmark environments, their tools, policies, simulators and evaluator internals; supplied environment clients and callbacks; and external data directories used by schema scripts. Their SkillX-side adapters and calls are included. The exclusion prevents conclusions about those dependencies' internal behavior, actual service availability, benchmark correctness or a runnable end-to-end deployment from this repository alone. Dependencies are visible in SRC-1, llm/client.py, clustering/embedding.py, inference/embedding_service.py, expansion/env_worker.py, expansion/env_explorer.py, scripts/extract_appworld_schemas.py and scripts/load_bfcl_schemas.py. No external dependency was separately acquired or executed.

The immutable evidence revision is b0047c848dbbcf019363d6d56eef8c558f69aefc, verified against the acquired checkout. The applicability cutoff is 2026-10-14, the supplied run date, not an assertion about when every bundled artifact was generated. Overall evidence is code-grounded: source inspection supports executable structure; source-native documentation and prompts support intent. No new target execution or causal experiment was performed. README performance claims remain attributed reports, not established gains (SRC-1, README.md).

The repository also freezes historical result summaries under exp_log/. Initial inspection of one telecom SkillX result found configuration and reward records but an unknown producing commit. These artifacts are eligible evidence of recorded results under their own stated conditions; they do not establish that this reviewed implementation produced them or isolate SkillX's causal contribution (SRC-1, exp_log/tau2-bench/skillx/2026-01-18T23_23_15.297811_telecom_llm_agent_Qwen3-32B_user_simulator_gpt-4.1-2025-04-14_test.json).

"info": { "git_commit": "unknown", "num_trials": 4, "max_steps": 200, "max_errors": 10, "seed": 300, --- exp_log/tau2-bench/skillx/2026-01-18T23_23_15.297811_telecom_llm_agent_Qwen3-32B_user_simulator_gpt-4.1-2025-04-14_test.json

Top-level coverage

Every top-level tracked file or directory is classified below. All rows refer to SRC-1. “Included” fixes functional scope, not a claim that every line has already been read. Initial inspection traced the principal production, maintenance, persistence and consumption paths and searched shipped Python callers and evaluation hooks. Available files awaiting deeper inspection are not access gaps. No material shipped path was inaccessible.

Tracked __pycache__/ files within included directories are excluded as duplicate compiled artifacts. Their exclusion prevents bytecode-specific or Python-version compatibility conclusions; the authored Python paths remain included. Directory inventories and caller searches establish their location, not behavioral equivalence with the reviewed source.

Source register