ai-modules synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
Install procedures, then let the host execute them
ai-modules is an agent operating extension layer, not an enclosing agent runtime. It supplies installable development, knowledge-management and editorial procedures, helper roles, scripts, hooks and evaluators. This analysis covers the shipped system at commit 0cf0227c416f75a79d7317f603a209a63e7a3c8f, with an applicability cutoff of 2026-10-14. Its sole source, SRC-1, provides inspected implementation and doctrine/design. No target, test, helper or provider was executed, and no retained execution trace was used. The whole-system functional boundary includes shipped callers and maintenance paths; it does not imply exhaustive inspection of every family. Host internals and live installations are excluded (boundary, Source register).
The architectural division is consequential. A host model interprets natural-language skill and agent bodies and chooses the next action. Host tools perform file and shell effects. Bundled Bash/Python implements narrower operations such as deployment, hashing, lint matching and evaluation. Copying or transforming a role into host configuration is wired; successful loading, callability, isolation and obedience are not demonstrated. Inherited model choices and evaluator aliases do not pin immutable model versions or establish parameter updates (RT-OBJ-skill-bodies, RT-RTE-deploy-artifacts, RT-CLM-callability-distinction, RT-CMP-host-model).
The claimed payoff is explicit retained intent and knowledge that later sessions can read rather than reconstruct. The source supports the admission and consumption paths, not measured accuracy, cost savings or convergence (RT-CLM-persistent-context).
Prepare a task through separate judgments
The representative workflow is task_auto_check: make one existing task implementation-ready without implementing it. It is selected because one procedure connects delegation, retained inputs, proposal rejection, controlled application, recovery and delivery. The foreground agent owns scheduling, writing and recovery. A missing spawn mechanism stops before freeze; generic orchestration's inline fallback does not override this family-specific boundary (RT-RTE-readiness-repair, RT-RTE-helper-dispatch, workflow account).
The procedure freezes task bytes and intent, clears the task-stem relay directory, writes pointer-only prompts and saves completed helper returns verbatim. A drift helper checks intent broadening. A gate applies task-check rules and owns readiness status. Stance reviewers propose issue-specific repairs. A proposal verifier rejects, narrows or approves exact replacements. The foreground applies approved changes after a fresh baseline comparison; a delta verifier checks what actually changed. Drift classification is narrower than whole-task fidelity checking. Human attestation can authorize broadening, while deletion or narrowing of frozen commitments and unresolved structural choices return to the user. These are different judgments, not votes against an independent known-good task (RT-RTE-readiness-repair, workflow account).
The control is a model-followed protocol, not a locked transaction. A fresh read does not prevent a concurrent write between comparison and application. Failed delta audits prescribe task-local restoration and one reapply/audit retry; failed or unassessable delta invocation takes a different helper-failure stop. Ordinary invalid helper results receive bounded identical retries, never improvised returns. Mechanical lint cleanup is separately authorized and some stops intentionally skip it. Citation challenges apply only at specified ready signatures; not every ready judgment receives that additional check. The default five-round cap has a bounded last-call citation/re-gate exception, not unlimited repair (RT-RTE-readiness-repair, workflow account).
Delivery is a task or stop report with status, counts, approvals/rejections, verification evidence and relay path. It does not implement, audit, archive, commit or release the described software. A later user-requested implementation reads the durable task and dependencies, edits code, runs verification and stops before user-owned audit/finish. Task audit then has a narrower evidence-consuming completion criterion: actual code, every body/Acceptance item, required tests and the design-extension signal. Its audited stamp licenses close-out under the protocol unless concrete code drift requires rechecking; it is not general program correctness (RT-RTE-task-implement, RT-BAP-task-guidance, EPI-RTE-task-audit, EPI-RTE-task-stamp, MEM-RTE-task-close).
Retain knowledge without conflating storage and warrant
The wiki keeps captured raw bodies, origin/digest metadata, typed pages, writing conventions, catalog summaries and action/audit history in distinct operative roles. Natural-language pages supply evidence or procedural guidance to later models; symbolic digest and lint fields have fixed checking consumers. Ordinary retrieval is request-conditioned: orient on schema, index and a bounded recent log, select relevant pages, and add recursive keyword search at 100 or more pages. No global token cap or fixed page count is established. Broad maintenance requests a separate context; actual isolation remains the host's responsibility (MEM-OBJ-raw-body, MEM-OBJ-raw-frontmatter, MEM-OBJ-wiki-pages, MEM-OBJ-index, MEM-OBJ-wiki-log, RT-RTE-wiki-consume).
Admission depends on the entry route. Import captures raw material before human page selections; session wrapup waits before any wiki write. Direct hub ingest, first query synthesis and safe deterministic maintenance can admit content without per-page approval. First synthesis is filed at medium confidence while validation is pending. Human selection, source attribution and confidence do not constitute truth acceptance. Acquired source warrant remains unknown, and no paired semantic input/output establishes whether cross-page composition preserves, entails or adds claims. Thus this witness supports candidate production and retention, not accepted ampliative knowledge or its subsequent lifecycle integration (RT-RTE-wiki-triage, EPI-RTE-raw-capture, EPI-RTE-wiki-compose, EPI-RTE-wiki-retain).
Maintenance detects disagreements and marks both pages contested without choosing a winner. Those flags warn later readers; they do not enforce retrieval exclusion. Archive removes active navigation while retaining bodies, and recursive searches are not established to exclude them. Raw preservation is also conditional: hub re-ingest replaces changed bodies after comparison, while maintenance contains conflicting general read-only and source-drift re-recording instructions. A refreshed digest certifies neither truth nor retention of earlier versions (EPI-RTE-wiki-critic, EPI-RTE-wiki-dispute, EPI-RTE-wiki-archive, MEM-OBJ-raw-body).
A concrete feedback mechanism is retained lint acceptance. The maintenance model may record an intentional info finding in schema policy; fixed matching removes matching info from later live reports, never warn/blocking findings. Audit baselines separately guide later cold-read working sets, with full fallback when the baseline is unusable. These mechanisms change checking workload by design, without establishing correct suppression or reliable semantic invalidation. Linter exit zero means no reported blocking issue within configured checks, not no warnings or true content (MEM-RTE-finding-acceptance, EPI-RTE-lint-acknowledge, MEM-OBJ-wiki-log, EPI-RTE-lint-dispose).
Evaluate artifacts, not infer causal improvement
The shipped wiki evaluator stages independent scenario directories, launches model workers, captures output/provenance/snapshots, then normalizes, grades and aggregates. Directory separation is not preventive security confinement: vendor launch requests permission bypass or a disabled sandbox. Normalization can discard response context while preserving report claims. Report-field equality checks what the worker reported, not whether it did the action. Artifact predicates license only their inspected scope (RT-RTE-evaluator-worker, EPI-RTE-report-normalize, EPI-RTE-scenario-check).
Final scenario success adds worker and optional integrity gates, but assertion averages remain ungated and missing integrity data is permissive. Aggregation compares prior benchmarks and signals regressions; a nonregressing failure need not yield process failure. These are implemented conditional deductions over artifacts, not observed pass rates, causal component effects or automatic product-change admission. Standalone compression/editorial skills likewise offer source-fidelity contracts and self-checks, not independent reconstruction evidence or a wired retained checkpoint (EPI-RTE-scenario-dispose, EPI-RTE-benchmark-rollup, EPI-RTE-text-check, MEM-CLM-summary-transfer).
Feedback contribution and independent unresolved properties
The strongest supported contribution is a set of durable feedback-to-consumer paths. Lint feedback can become persistent checking policy, helper criticism can become a revised task contract for later implementation, and operational patterns can become reusable trigger/action/reason procedures. The first two automatic trace-fed writes are claimed model procedures; fixed downstream lint matching is wired. Procedure and completed-task lesson extraction afford additional trace-fed use. This is more than raw logging, but it does not demonstrate improved capacity for future action. Manually selected session extraction has trace origin without being an automatic learning write (MEM-RTE-finding-acceptance, RT-RTE-readiness-repair, MEM-RTE-procedure-capture, MEM-RTE-task-close, profile verification).
The task-repair witness leaves theory-builder condition 1, a selected localized theory, uninspected; condition 2, decisions depending on its content, uninspected; and condition 3, content-directed criticism with blame and changed reliance, uninspected. Condition 4 has a claimed repair-to-next-gate iteration, but theory-specific iteration is not established. Exact anchors make task text addressable without proving these other conditions. Whether criticism of a consumed theory improved future capacity remains uninspected. Reflection likewise remains uninspected: representing work is not sufficient evidence of a two-way self-representation. Computational ownership of proposal/checking substeps is positive, but autonomous or reflective theory building is unestablished (RT-RTE-readiness-repair).
Self-improvement is independently unestablished over the inspected cross-invocation horizon. Lint feedback offers a standing policy-update pathway directed at avoiding repeated justification, with a durable consumer capable of changing later reporting. No subsequent operation was inspected that exercised and causally depended on that change; successful improvement has no outcome comparison either. Task repair may improve an external work contract without establishing change to the extension's own organization. None of these gaps establishes system-wide absence of learning, reflection, autonomy or self-improvement (MEM-RTE-finding-acceptance, EPI-RTE-lint-acknowledge, RT-RTE-readiness-repair).
For one-task preparation and later file-backed knowledge work, the distinguishing capacity is explicit retained state plus separated checking responsibilities. Its reliability depends on model compliance and host contracts. Live helper traces, paired source/page transformations and repeated policy-consumption runs would change the operational assessment. Independent outcome comparisons would change benefit attribution. Host-enforced tool restrictions, atomic application controls, explicit resolution of raw-maintenance precedence and evidence-consuming claim acceptance would change the corresponding guarantees. These are assessment-changing evidence or system changes, not adoption recommendations.
Limitations
| limitation | affected SRC IDs and record citations | inspected boundary | conclusion prevented | evidence that would resolve it |
|---|---|---|---|---|
| Grouped comparison units are not an incidence matrix; this carries the profile verifier's declared limit. | SRC-1; MEM-OBJ-raw-body, MEM-OBJ-task-files, EPI-OBJ-test-report; profile verification Limits | Accepted partial storage, form, lineage, authority and admission inventories; optional repository history and mixed evaluation parts have local warrants. | Per-object incidence counts or cross-axis combinations inferred solely from shared unit membership. | An expanded object-by-consumer inventory preserving each finding's local evidence basis. |
| Functional whole-system scope exceeds detailed inspection of some families, assets and initial provenance. | SRC-1; boundary coverage, MEM-OBJ-raw-body, MEM-OBJ-wiki-pages, MEM-OBJ-task-files, report verification | Traced task/wiki/evaluator paths; remaining shipped-family output consumers, image consumption and preexisting instances remain partial or uninspected. | Exhaustive mechanism enumeration, complete-value statistics, corpus truth or system-wide negative findings. | Recorded inspection of unresolved material operations, assets and instance provenance at the same boundary. |
| No operation or benefit witness was used. | SRC-1; RT-RTE-readiness-repair, RT-RTE-wiki-consume, MEM-RTE-finding-acceptance | Implementation and doctrine/design only. | Successful delegation/recovery, actual later influence, improved capacity, cost savings or causal effectiveness. | Provenance-linked execution and later-consumer traces; controlled comparisons for benefit and component attribution. |
| Host callability, context isolation, permissions and model adaptation are external or opaque. | SRC-1; RT-CMP-host-model, RT-RTE-deploy-artifacts, RT-RTE-helper-dispatch, RT-RTE-charter-protect | Emitted configuration, prompt contracts and best-effort hook scan, not live hosts. | Universal compatibility, enforced read-only execution, complete charter protection, immutable model identity or parameter-update conclusions. | Inspected host enforcement/loading contracts and live traces, including indirect mutation and alternative channels. |
| Raw-maintenance permission has conflicting instructions; prior-version retention is not guaranteed. | SRC-1; MEM-OBJ-raw-body, MEM-OBJ-raw-frontmatter, EPI-RTE-raw-capture | Hub re-ingest plus maintenance's read-only constraint and re-recording exception. | Categorical raw immutability, unrestricted maintenance replacement or recovery of every former source version. | Explicit instruction precedence, provenance-linked refresh traces and a version-retention contract. |
| Semantic generation and reliance are not truth acceptance. | SRC-1; EPI-RTE-wiki-compose, EPI-RTE-wiki-retain, EPI-RTE-wiki-dispute, EPI-RTE-text-check | Pending synthesis, model semantic checks and advisory dispute flags. | Entailment/novelty classification, accepted ampliative knowledge, lossless maintenance or independent reconstruction guarantees. | Paired semantic inputs/outputs and recorded evidence-consuming acceptance against named criteria, use and scope; independent receiver checks. |
| Retained audit policy changes workload without proven correctness or universal exclusion. | SRC-1; MEM-OBJ-wiki-log, EPI-RTE-lint-acknowledge, EPI-RTE-wiki-archive | Incremental cold-read selection, info-only suppression and navigation withdrawal. | Reliable semantic invalidation, correctly judged intentionality, full corpus cold-reading or complete archived-content access revocation. | Repeated audit traces covering baseline changes and suppression decisions, plus explicit tested archive-search behavior. |
| Evaluator evidence and process exits have narrower meanings than worker compliance or confinement. | SRC-1; RT-RTE-evaluator-worker, EPI-RTE-report-normalize, EPI-RTE-scenario-check, EPI-RTE-scenario-dispose, EPI-RTE-benchmark-rollup | Inspected launch/normalize/grade/aggregate code; no scenario execution evidence assessed. | Actual pass rates, complete I/O provenance, security confinement, test completeness or causal product improvement. | Complete worker/tool traces, inspected scenario fixtures and comparisons, preserved original responses and independently checked integrity evidence. |
| Theory-specific and self-improvement attribution remains unresolved independently of positive feedback routes. | SRC-1; RT-RTE-readiness-repair, MEM-RTE-procedure-capture, MEM-RTE-finding-acceptance | Fixed task-repair methodology and retained procedural/checking outputs; no selected theory or exercised self-change witness. | Established theory-builder conditions 1–4, reflective/autonomous theory building, theory-criticism learning or exercised self-improvement. | A localized consumed theory, content-directed criticism with blame and retained next-round consequence; two-way self-representation evidence; objective-linked self-change followed by causally dependent later operation. |