SkillWeaver synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
SkillWeaver's strongest supported contribution is a connected route from browser experience to durable changes in its own later action machinery. It synthesizes reusable Python functions, retains their documentation and metadata, and supplies them to later task attempts. A second route changes future practice and visibility through persistent test counters. Both are implemented trace-fed behavior-shaping writes, not demonstrated gains in capacity. The unresolved question is whether subsequent operation depends on those changes and performs better because of them (RT-RTE-skill-admission, RT-BAP-library-execution, EPI-RTE-tested-label). Evidence: SRC-1, skillweaver/knowledge_base/knowledge_base.py, skillweaver/agent.py, skillweaver/explore.py.
The evidence is source inspection of the canonical SkillWeaver repository at commit f2a63d65d0f6ff46ac30e817cede8797f8f25b97, with an applicability cutoff of 2026-10-14. The boundary classifies the target as a whole-system builder or improvement plane: exploration, native browser actions, skill admission and persistence, the experimental Browser-Use adapter and benchmark drivers are included. Code supplies implementation evidence; README and prompts supply intent; shipped skill usage logs supply reported operation only. No execution trace or causal comparison is registered. External model services, browser implementations, website/container state and Browser-Use planning are not frozen. The accepted report verification and profile verification have no remaining blockers or declared limits; that does not remove the records' evidence boundaries. Source identity and layers are registered as SRC-1 in the Source register.
Practice supplies the update evidence
A representative invocation is python -m skillweaver.explore reddit <out_dir>. The caller supplies website, models, iteration budget, schedule and optional initial library. Asynchronous workers own browser wrappers but share the knowledge base. Software selects an untested function for practice or asks a model to propose a website task. Locks serialize selection and admission, not entire attempts; another worker can change the shared library while an attempt proceeds. These are not independently isolated delegated agents (RT-RTE-explore-practice, workflow account). Evidence: SRC-1, skillweaver/explore.py.
Each native attempt starts fresh action/state lists and disabled names. It selects skill descriptions once, then asks a model for Python actions or termination. Each action sees the current browser observation and formatted earlier actions, rather than every saved screenshot and intermediate tree. Separate relevance, success and synthesis calls receive different selected inputs. This is code-owned working-context selection, not a durable conversation summary. Generated Python is imported and executed in the host process with the full library unless reference-only mode omits it. Selected descriptions and hidden unverified names therefore are presentation controls, not a selected-only capability boundary. Five-response generation retries are best effort, not a final invariant veto (RT-RTE-native-task, MEM-OBJ-traces). Evidence: SRC-1, skillweaver/attempt_task.py, skillweaver/agent.py.
The attempt returns states and actions. The builder's default text rendering retains the initial observation and action/result fields but omits intermediate state renderings and explicit recovery annotations. It overwrites the task driver's richer readable trajectory; raw artifacts still survive separately. The success model receives this reduced text plus the final screenshot. Synthesis receives the reduced text without that separate screenshot or the success model's reasoning. Thus saved experience is richer than the evidence delivered to the update consumer (EPI-RTE-trajectory-format, MEM-OBJ-traces). Evidence: SRC-1, skillweaver/agent.py, skillweaver/explore.py.
Two branches confer different kinds of reliance
For a test, a matching named-call node anywhere in the last action's source, together with no recorded exception or recovery attempt, triggers a counter increment and bypasses success judgment and synthesis. A call under if False can satisfy this predicate without executing the skill. The stored label therefore establishes neither exercised skill use nor website success. Nevertheless, its counter affects later visibility, synthesis inclusion and practice scheduling. Loaded metadata and an exposed blanket-marking method can also supply tested status without replaying tests (EPI-RTE-test-check, EPI-RTE-tested-label, MEM-OBJ-metadata). Evidence: SRC-1, skillweaver/attempt_task.py, skillweaver/explore.py, skillweaver/knowledge_base/knowledge_base.py.
Other attempts reach the success model. Its Boolean is a task-scoped inference from visible evidence, not an independent expected-outcome oracle. Software consumes a positive judgment to accept the episode as synthesis input; a negative judgment withholds synthesis. This acceptance does not establish the correctness of a not-yet-generated generalized procedure (EPI-RTE-success-check, EPI-RTE-success-decision). Evidence: SRC-1, skillweaver/knowledge_base/check_success.py, skillweaver/explore.py.
Installation makes skills durable, not warranted
Synthesis receives task, selected trajectory, semantic text, update instructions and filtered existing function source. Prompts request generalization and usage documentation. Static checks report coding violations and can request up to ten proposals, but exhaustion still reaches application; parse/format errors can instead stop it. Same-name functions replace existing bodies and documentation, omitted names remain, and changed functions' test counts reset. Neither docstring presence nor navigation-text occurrence proves documentation truth, safe replay or website outcomes. Actual generated content may reshape or generalize its inputs; its semantic relation is indeterminate without candidate/input instances. This is operational installation, not integration of an accepted generalized claim (RT-RTE-skill-admission, EPI-RTE-code-check, EPI-RTE-skill-merge). Evidence: SRC-1, skillweaver/knowledge_base/knowledge_base.py, skillweaver/knowledge_base/code_verification.py, skillweaver/templates/kb_procedural_update_base.md.
Delivery consists of live shared-library mutation and an iteration checkpoint containing code, metadata and semantic text. A checkpoint can also follow a negative success judgment; exceptions before saving can prevent delivery. Diagnostic handling is not transactional rollback, manual prefix reload is not automatic best-checkpoint recovery, and the CLI's zero exit does not prove successful exploration (workflow account, RT-RTE-explore-practice). Evidence: SRC-1, skillweaver/explore.py.
Later tasks consume two distinct retained parts: Python bodies execute when invoked, while signatures/docstrings guide relevance selection and model action choices. Ordinary reload restores these parts and metadata, not synthesis reasoning from diagnostics. Loaded semantic text informs proposals and synthesis but is unchanged by the inspected update. Established consumer chains request retained material; optional embedding selection, schema exports and artifact reconstruction remain capabilities without established later-consumer routes. They cannot establish complete pull-only coverage or a push signal (MEM-OBJ-code, MEM-OBJ-docstring, MEM-OBJ-semantic-knowledge, MEM-RTE-embedding-selection, profile verification). Evidence: SRC-1, skillweaver/knowledge_base/knowledge_base.py.
Recovery, alternate consumption and assessment
Optional locator recovery can check a model-proposed replacement against type, count and selected-element identity, then retry the original operation. It does not prove user intent, roll back earlier effects or directly repair retained code. Browser-Use instead receives selected callbacks constructed after executing library code; its external planner owns continuation and termination, without inheriting native checking or exception tracking (RT-RTE-locator-recovery, RT-RTE-browser-use). Evidence: SRC-1, skillweaver/environment/patches.py, skillweaver/attempt_task_browser_use.py, skillweaver/knowledge_base/knowledge_base.py.
For bounded website practice, the source supports automatic proposal, execution, episode judgment, installation and later availability. For generalized task correctness or improved future performance, it supplies no observed or causal conclusion. Benchmark checking is a separate path: inspected string/URL rules assess configured representations, and scores feed reporting rather than skill acceptance or rollback. A missing unconditional computer-use import and CLI mismatches obstruct a clean-source evaluation claim, although local scoring code exists (RT-RTE-benchmark-evaluation, RT-ABS-cua-module, EPI-RTE-benchmark-check, EPI-RTE-score-retention). Evidence: SRC-1, skillweaver/evaluation/evaluate_single_task.py, skillweaver/evaluation/evaluate_benchmark.py, skillweaver/evaluation/webarena_evaluators.py.
The properties remain separate. The inspected builder is computationally autonomous in its internal practice/update roles, without per-skill human approval; this says nothing about reliability or autonomy of an unestablished theory-building loop. Theory-builder conditions 1–4 are not individually established by the supplied runtime account: generalized procedures, delivery, checker feedback and repetition cannot substitute for demonstrated localized theory content, content-dependent decisions, formulated content-directed criticism and retained criticism shaping another round. The records likewise do not establish that criticism of a consumed theory improved future capacity, or that the system is reflective. These are unestablished findings, not evidenced absences. In contrast, the builder-plus-consumer boundary supports an improvement-directed dispositional self-change path, with the objective of reusable website automation over later iterations/tasks. It does not establish exercised self-improvement: no trace closes subsequent causal dependence on an update, and no comparison establishes successful improvement. Provider weight changes remain independently uninspected (RT-RTE-skill-admission, RT-CLM-self-improvement, RT-CMP-lm, epistemic conclusion). Evidence: SRC-1, skillweaver/knowledge_base/knowledge_base.py, skillweaver/explore.py, skillweaver/lm.py.
An update-linked execution trace would change the exercised-operation assessment. A controlled library intervention and comparable tasks could establish benefit at the intervention's actual scope. Runtime call instrumentation and authoritative outcome checks would strengthen tested labels; a final checker veto would change admission strength but still would not prove website semantics. Candidate/episode comparisons and generalized behavioral tests would clarify what synthesis produces and licenses. These are evidence or system changes needed for stronger conclusions, not claims about operation at this pin.
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No registered execution or intervention/comparison evidence | SRC-1; RT-RTE-skill-admission, RT-BAP-library-execution, RT-CLM-self-improvement | Frozen builder and later native/controller consumers | Observed delivery/use, exercised causal self-improvement and increased future capacity | Update-linked traces and controlled comparisons with independently varied treatment |
| Arbitrary initial content and override admission have unresolved provenance | SRC-1; MEM-OBJ-code, MEM-OBJ-docstring, MEM-OBJ-semantic-knowledge, MEM-OBJ-metadata | Loader, shipped samples and exposed metadata controls | Attribution of all content to autonomous practice, complete lineage or human approval | Original producer/admission records and route-linked input artifacts |
| Delivered episode omits later observations and recovery annotations | SRC-1; MEM-OBJ-traces, EPI-RTE-trajectory-format | Native rendering and builder overwrite; separate final screenshot to success only | Complete update-evidence lineage or guaranteed durable learning from locator repair | Consumer-level traces of the exact prompts and repair information retained in later skills |
| Generalized efficacy and documentation truth are not checked by installation | SRC-1; EPI-RTE-success-decision, EPI-RTE-code-check, EPI-RTE-skill-merge | Successful-input branch, bounded repair and name-local merge | Accepted generalized knowledge, safe replay or determinate semantic novelty | Candidate/input comparisons and evidence-consuming behavioral acceptance against a named domain |
| Tested status has only source/result-field evidence and alternative overrides | SRC-1; EPI-RTE-test-check, EPI-RTE-tested-label | Last-action AST predicate, counters and filtering | Dynamic skill execution, outcome verification or universal executable revocation | Call instrumentation, outcome checks and authenticated metadata provenance |
| Optional APIs lack established later-consumer routes; visual form remains unresolved | SRC-1; MEM-RTE-embedding-selection, MEM-OBJ-docstring, MEM-OBJ-traces, profile verification | Selection/export/reconstruction capabilities and retained screenshots | Complete direction/signal inventory or complete controlled form classification | Actual later-consumer wiring and evidence of visual-payload consumption |
| External implementations and exact provider versions are not frozen | SRC-1; RT-CMP-lm, RT-CMP-embed, RT-RTE-browser-use, boundary | Client adapters, browser/site dependencies and controller handoff | End-to-end reproducibility, host isolation, hidden retention/learning or external planner guarantees | Versioned dependency/provider contracts, environment captures and inspectable executions |
| Benchmark entry defects and unassessed skill/evaluator alternatives | SRC-1; RT-RTE-benchmark-evaluation, RT-ABS-cua-module, EPI-RTE-benchmark-check, report verification | Shipped drivers, inspected string/URL rules, sampled libraries; other bodies available but not fully assessed | Working clean-pin evaluation, all-skills correctness or complete evaluator-warrant comparison | Resolved entry dependencies/options, broader relevant source inspection and resulting run artifacts |
| No established formulated criticism or reflective causal loop | SRC-1; RT-RTE-skill-admission, epistemic conclusion | Procedural update instructions, checker repair and retained documentation | Theory-builder conditions, reflection or improved capacity attributable to theory criticism | Formulated theory/criticism records, their consequential revision and next-round consumers; separate improvement evidence |
Source-read log: none. This synthesis uses the supplied accepted records and adds no frozen-source evidence.