SkillX synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From supplied experience to reusable guidance
SkillX implements automatic experience-to-instruction writes: selected histories feed model-generated plans and skills, software admits library entries, and saved content can be reloaded for later prompt preparation. This is its strongest supported contribution. Whether those changes improve capacity for future action remains unresolved; neither the implementation nor the available historical artifacts establishes that benefit. The account covers the shipped repository at commit b0047c848dbbcf019363d6d56eef8c558f69aefc, with applicability cutoff 2026-10-14. SRC-1 supplies implementation, prompts and documentation, plus bounded samples of bundled artifacts. No new target execution or causal experiment was performed. External model providers, embedding servers and benchmark environment/evaluator internals are outside the frozen evidence. See the boundary and Source register, RT-RTE-library-build and RT-RTE-prompt-prepare.
Architecturally, SkillX is a memory/knowledge/context-engineering system with an offline construction plane, separate exploration loops and a retrieval-to-prompt service. Its reusable products are task plans, functional reference implementations and atomic tool-use examples. They are not installed callable functions. Library JSON preserves text and symbolic access metadata; inference derives in-process vectors for similarity access. Generation and embeddings use separate opaque model dependencies, without frozen weight identities or inspected parameter-update operations. Changing retained instructions is distinct from changing model weights. See MEM-OBJ-plan-skill, MEM-OBJ-functional-skill, MEM-OBJ-atomic-skill, MEM-OBJ-retrieval-vectors, RT-CMP-llm and RT-CMP-embedding-service.
The representative workflow is one-epoch functional-skill construction without expansion. A caller supplies trajectory files and configuration; Python code owns stage scheduling. It selects stored rewards strictly above 0.999 by default, summarizes long user/tool feedback, selects the shortest successful history per task, requests plans and then step-specific skills. Models receive separate stage-specific message frames, not the enclosing agent conversation. Plans enter the library before skill filtering. Enabled filters judge candidates, embeddings cluster survivors, and a model merges multi-member clusters before conversion and library admission. Delivery means saved library/checkpoint files, statistics and an API return—not tested task success. The workflow account establishes this continuous path without joining unrelated exploration and inference capabilities into a fictitious completed cycle; its canonical route is RT-RTE-library-build.
Context recurrence and admission have different boundaries
SkillX already changes later extraction context before final library admission. Functional extraction supplies its augmented proposal list to subsequent plan steps. Atomic/hybrid extraction selects prior tool proposals from an extractor-instance map, not from the pipeline's prepared committed-library field. Filtering and final replacement do not synchronize that map or retract rejected proposals. This is provisional recurrence, not guaranteed feedback from the committed epoch library. Long-feedback summaries likewise replace text in a copied history for later extraction; the original caller input remains available, but there is no designated pointer-based reload protocol. The trigger is character length, not token occupancy, and summary fidelity is unchecked. See MEM-RTE-proposal-context and MEM-RTE-feedback-summary.
The filters have real operational force: their Boolean results exclude covered candidates. Their intended criteria concern reusable text and tool-use compatibility with supplied schemas, not demonstrated execution or transfer. Parsing is weaker than the requested judgment: not good satisfies the general filter's substring test, and incorrect satisfies the schema filter's correct test. Atomic-category candidates bypass the general stage; entirely missing usable schemas can return unchecked survivors; caller configuration and direct library APIs provide other admissions. Default pre-merge filtering does not check the generated merged successor. Functional content can also be truncated at its first return after that check. These facts bound the gate rather than proving harm to a particular skill. See EPI-RTE-general-check, EPI-RTE-schema-check, EPI-RTE-filter-disposition, RT-CLM-quality-control and workflow annotations.
Final retention is selective about reasons. Fixed serialization keeps skill content and metadata, not the outer modification rationale, filter discussion or raw input histories; epoch files retain statistics rather than full candidate transcripts. Same-name replacement, incoming atomic redundancy suppression and caller-selected snapshots offer narrower revision/recovery mechanisms, not global withdrawal or transactional rollback. Renamed successors can coexist with predecessors, and existing atomic entries are not globally purged. Standalone atomic extraction can be collected as functional, unlike the hybrid atomic branch. The optional checked-export preflight is statically blocked by an embeddings_config/embeddings constructor mismatch; standalone structural predicates do not repair it, and unchecked save/export remains available. See MEM-OBJ-skill-library, MEM-OBJ-skill-metadata, MEM-OBJ-atomic-skill and EPI-RTE-export-validation.
Later requests reach prompt preparation, not a complete deployment
A caller can reload a unified library and request guidance for a task or step. Cosine similarity ranks plans and skills; tool eligibility and optional model selection narrow delivery. Limits bound item counts, not total prompt tokens. An empty tool set is falsy and does not enforce zero eligible tools. The inference service is requested retrieval; automatic proposal/summary supply inside construction is a different later-consumer mechanism. Embedding failures can produce cached zero vectors that suppress positive-threshold matches without emptying the store. See RT-RTE-prompt-prepare, MEM-OBJ-retrieval-vectors and MEM-RTE-proposal-context.
The actual prompt boundary matters. The service's rewrite-enabled call has a required-argument mismatch, and its selector's generate interface does not match the shared LLM, with first-candidate fallback on selection failure. BFCL drops both plan and base prompt; tau2 drops the plan while retaining base policy; AppWorld's added human-skill metadata does not enter the already-formatted prompt. All three benchmark wrappers leave task execution unimplemented. A separate public rewrite helper correctly passes plural references and affords rewriting, but no shipped caller or later delivery is established. A separate self-filter likewise affords selection without repairing the service selector. Thus metadata availability is not delivered guidance, and prepared reference text is not exercised downstream compliance. See RT-RTE-prompt-prepare, RT-BAP-skill-guidance and MEM-RTE-self-filter.
Exploration does not close the improvement cycle
Separate rollout implementations let models choose actions while code controls local stopping. Generic callbacks receive assistant text; AppWorld executes extracted Python and calls an external evaluator; a client explorer can forward provider-native tool calls and consume host done/reward signals. These are distinct effect and warrant routes. Generic completion strings establish much less than task evaluation, and even a correct task outcome does not validate a reusable procedure or its transfer. Passed API-avoidance checklists are not enforced by the two worker bodies. Local step budgets do not bound nested callback effects or establish deployed isolation. In addition, response parsers can evaluate provider-controlled Python expressions locally before type checking; that immediate process effect is separate from advisory skill delivery. See RT-RTE-environment-rollout, RT-CLM-rollout-control and RT-RTE-response-eval.
Expansion can propose objectives from real or simulated traces and checkpoint generation progress. Simulated observations and model confidence are not environmental evidence. The convenience pipeline leaves its worker unset, a task manager alone does not activate expansion, and the original explorer expects a chat interface absent from the shared LLM. Converted objectives have empty histories and reward=None; the pipeline does not execute them before the next success comparison. Repeated cumulative duplicate filtering can remove previously admitted queries. These are supported generation/recovery capacities with integration breaks, not a closed automatic execution-to-extraction cycle. See RT-RTE-task-synthesis and MEM-OBJ-task-checkpoint.
What the supported change does—and does not—establish
For offline instruction construction, the code provides a continuous producer-to-file workflow with partial-result recovery: retries, item skipping and first-member merge fallback. For task execution, it provides prompt preparation and separately exposed rollout capabilities, not a complete shipped benchmark deployment. For epistemic reuse, checks grant bounded operational admission against text/schema criteria; retention and relevance ranking do not establish accepted generalized knowledge. Semantic relations in model abstraction, compression and merging remain indeterminate without traced input/output comparisons. README quality and transfer gains remain attributed claims, not a controlled comparison at this commit. See RT-RTE-library-build, EPI-RTE-filter-disposition, EPI-CLM-quality-refinement and EPI-CLM-transfer-gains.
The records establish trace-fed instruction-update capacity, not demonstrated learning as improved future capacity. They do not establish a formulated-theory route on which to assert theory-builder condition 1 (localized theory content), condition 2 (decisions dependent on that content), condition 3 (content-directed criticism with blame and revision), or condition 4 (that criticism's result shaping another round). Each remains unestablished independently; opaque model processing is not evidence of its absence. Criticism of a consumed theory improving future capacity is likewise unestablished. Reflection is not established merely by maintaining skills, and automated stage scheduling does not establish autonomous theory building or an exercised autonomous improvement cycle. These conclusions concern the assessed routes, not every possible generated artifact. See RT-RTE-library-build and the epistemic bounded conclusion.
At this boundary, the standing update-and-guidance path supports only a prospective, dispositional self-change account: experience can shape retained behavior-relevant instructions and those instructions can reach later context. Exercised self-improvement is unestablished because subsequent live operation causally dependent on the update is not supplied; successful improvement additionally needs outcome evidence against an independent objective. No inspected component intervention attributes the reported gains to SkillX or a particular mechanism. Linked construction/admission traces, delivered prompts and later behavior would strengthen operation claims; controlled outcome comparisons would address benefit. Repairing rewrite/export interfaces, implementing wrapper execution and executing synthesized tasks before extraction would change the shipped integration assessment, but repairs alone would not prove learning, warrant or gains. See RT-RTE-library-build, RT-BAP-skill-guidance and EPI-CLM-transfer-gains.
Limitations
Both independent verifications report no blockers or additional publication limits, and the supplied reconciliation identifies no current unresolved conflict. The following rows retain the evidence boundaries that prevent stronger conclusions. Source-read log: none; this synthesis uses the handed members only.
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| Construction and later behavioral dependence are not observed at this commit. | SRC-1; RT-RTE-library-build, RT-BAP-skill-guidance | Shipped build, persistence and preparation code; no new execution | Successful delivery, learning as improved capacity and exercised self-improvement | Commit-linked traces connecting input, admitted update, actual prompt and subsequent behavior; outcome evidence against an independent objective |
| Input rewards, prior content and external evaluators have unresolved provenance or fidelity. | SRC-1; MEM-OBJ-trajectory, EPI-OBJ-tool-schemas, RT-RTE-environment-rollout | Supplied-file selection, schema consumers and host evaluator calls | Authenticated success, schema truth and reusable-procedure warrant from outcomes | Origin/evaluator records, versioned tool contracts and direct scoped procedure tests |
| Generated semantic relations and criticism history are not established. | SRC-1; MEM-RTE-feedback-summary, MEM-OBJ-skill-metadata, EPI-OBJ-skill-content | Model prompts, serializers and bounded candidate samples | Faithful abstraction, entailed or ampliative production, claim-specific acceptance and theory-builder properties | Linked source/output pairs, checked proposition relations and retained criticism/revision/consumer traces |
| Shipped delivery and admission paths contain implementation breaks and bypasses. | SRC-1; RT-RTE-prompt-prepare, EPI-RTE-export-validation, RT-CLM-quality-control, RT-RTE-task-synthesis | Service/wrappers, exporter, filters and expansion conversion | Universal validation, complete benchmark deployment or closed automatic improvement cycle | A new boundary with repaired interfaces, explicit execution wiring and observed covered-path results; semantic checks of final admitted content |
| Backend identity and deployed effect permissions are external. | SRC-1; RT-CMP-llm, RT-CMP-embedding-service, RT-RTE-response-eval, RT-CLM-rollout-control | Provider clients, local parsers and environment adapters | Frozen-model reproducibility, global safety/isolation or actual exploit occurrence | Immutable provider identities, service contracts, deployment grants/isolation evidence and relevant execution traces |
| Sample compatibility and memory inventory remain bounded. | SRC-1; MEM-CLM-format-boundary, MEM-OBJ-task-checkpoint, profile verification | Iteration-one shapes, known consumer routes; remaining payloads, optional adapters/parsers and individual schemas not exhaustively assessed | Complete artifact compatibility, complete memory classifications or system-wide absence findings; all ten profile axes remain partial | Recorded inspection of included alternatives, explicit conversions and established later consumers |
| Historical results do not identify this producing implementation or a controlled component effect. | SRC-1; EPI-CLM-transfer-gains, boundary | README reports and bounded bundled-result sample with unknown producing commit | Reproducible transfer/efficiency gains or causal attribution to SkillX's mechanisms | Revision-linked runs, comparable outcome aggregates and independent variation of the component being attributed |