EvoOntology synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From workload to reusable guidance
EvoOntology is a knowledge layer for data agents, not their enclosing autonomous runtime. This account covers its shipped construction, persistence, consumption and evolution paths, both host plugins, and the three included benchmark consumers at commit c94389b764394379086ff25e50d0e0f4396167b4, with an applicability cutoff of 2026-10-13. SRC-1 supplies inspected implementation, design instructions and attributed README performance reports. No target execution, reproduced benchmark result or causal experiment was supplied. External hosts, user data, provider services and datasets are outside the frozen evidence. Whole-system responsibility coverage does not mean exhaustive inspection; InsightBench's recovery/history inventory remains partial (boundary, Source register).
The system divides intelligence from enforcement. A host model follows Build/Evolve skills to probe data, propose meanings, diagnose failures and design patches. A deterministic Python core exposes semantic and lifecycle operations through MCP, stores state and vetoes particular invalid transitions. Shipped benchmark agents have their own model/tool loops. The core is neither a model trainer nor the host's permission manager (RT-CMP-host-model, RT-RTE-benchmark-tool-loop).
Construction starts from authorized workload questions and actual-data probes. The host proposes five retained families: terms define concepts; mappings connect concepts to physical data, filters and grain; relations describe connections; constraints express applicability and cautions; evidence records hold asserted sources, queries, results and validation methods. These are file-backed, mixed prose and symbolic records, not an embedding index. Their IDs and links support deterministic access; their meanings require model interpretation (MEM-OBJ-term, MEM-OBJ-mapping, MEM-OBJ-relation, MEM-OBJ-constraint, MEM-OBJ-evidence).
Build instructions require meaningful data correspondences, explicit scope and granularity, reproducible evidence, and revision of unsupported objects. Initial publication instead mechanically checks representation, references and loadability, refuses an already initialized active index, and selects the version. Thus the architecture supports grounding work and a declared semantic acceptance policy; it does not automatically certify the stored propositions. Evidence strings are not authenticated execution receipts, and no observed produced accepted ampliative claim is established (EPI-CLM-grounded-commitment, EPI-RTE-version-validation, RT-RTE-build-publication).
From retained versions to task execution
Later root semantic calls reload an active or explicitly requested version. Browse ranks lexical matches and returns at most six items; resolve processes the first five nonempty mentions and returns selected definitions, attachments and evidence bodies, including ambiguity and coverage markers. These limits constrain items, not serialized bytes or cumulative model context. Projection omits some stored qualification fields, including mapping database-source and validation metadata. A supported coverage marker reports attached content, not verified truth; a constraint severity of block supplies guidance, not a native-query veto (RT-RTE-semantic-retrieval, EPI-RTE-semantic-projection, RT-BAP-semantic-guidance).
Consumer differences change what can be relied on. BIRD drops lifecycle metadata and follows only some evidence-body links, so root withdrawal rules do not generalize to it. DDR's shipped manifest callers frontload financial concept references without passing the optional task argument; the task-ranking API and its 15-concept cap are available but not activated by those callers. DDR resolve supplies evidence IDs, not evidence bodies or the promised sample query. InsightBench provides table-bound guidance and transient reply deduplication, with an empty-resolve binding fallback; its Python resolve loop does not enforce the tool schema's five-mention limit. Binder correctness and shortcut filtering remain unestablished (MEM-RTE-bird-guidance, MEM-RTE-ddr-guidance, MEM-RTE-insight-guidance).
A recorded project task pins a concrete version name, resolves guidance, gathers native observations, and finishes into a retained trajectory. Built-in SQLite replay executes under read-only/action/progress limits and returns at most 100 rows. Completion requires at least one non-error native event, but manually supplied events are caller assertions. Neither that guard nor successful query execution establishes final-answer correctness. Direct semantic calls are not automatically task-recorded. The pinned version name is mutable, not a content digest (RT-RTE-task-execution, RT-RTE-version-override).
Retained experience is selective. Project events can truncate even semantic replies; BIRD/DDR normalization loses a unified cross-family order and truncates native results; InsightBench's normalized trajectories omit native calls. BIRD can label an explicitly selected candidate's experience with the active version reread at recording. BIRD/DDR translate full session history, whereas InsightBench's inspected ToolCallingChat creates a fresh frame for each call. Earlier stage transcripts enter only if supplied by callers; terminal text can cross back. Transient deduplication does not restore an earlier reply absent from a later frame. None of these mechanisms establishes external host compaction or a universal context budget (MEM-RTE-trace-retention, MEM-OBJ-trajectory, RT-RTE-benchmark-tool-loop).
From experience to candidate admission and delivery
The strongest implemented trace-fed update is history-to-workload reuse: compatible retained task experience is automatically reduced to durable questions/provenance, normalized and merged, then consumed by later preparation or task starts. Fixed-split preparation disables automatic history/generated input and checks construction IDs; rolling preparation filters source and split and checks generated-question purpose/provenance. This can change future task/build inputs without editing the ontology. It is a wired durable trace-fed update, not evidence of improved future capacity. User questions all survive selection, so the requested limit is not a hard total-count ceiling (MEM-RTE-workload-reuse).
Ontology revision is a separate model-mediated pathway. The representative Codex workflow begins with user-authorized objectives, unacceptable regressions, evaluation setup and round budget. A threshold reminder does not start it. The host freezes the Parent and input batch, diagnoses failures, attributes a primary mechanism, states a falsifiable hypothesis and creates a separate Candidate. It orchestrates Parent/Candidate evaluation under instructed fixed conditions. The three benchmark adapters run subprocess drivers and return normalized observations, but omit formal gate_input; the caller must pair results, assess regressions and record that input (RT-RTE-evolution-admission, RT-RTE-benchmark-evaluation, MEM-RTE-reminder-delivery, workflow account).
Session acceptance validates the Candidate and recomputes the current-round/current-Candidate gate. Paired ground-truth scores must be finite with unique matching case IDs and a higher candidate mean; judge verdicts require more candidate than parent wins and zero candidate critical errors. The caller must explicitly assert no unacceptable regressions, and the protocol must match when frozen. These are enforcing conditions on this acceptance API, not checks that the observations are authentic, independent or reproducible. BIRD's inspected SQL scorer licenses sorted-row equivalence on one database given the reference SQL; it does not prove query intent, causal explanation or transfer. Detailed DDR/InsightBench evaluator licenses remain unassessed (EPI-RTE-gate-aggregation, EPI-RTE-sql-comparison, RT-BAP-session-gate).
Rejection retains notes and candidate artifacts and returns control to search. Incomplete termination follows budget or reason-specific guards. Acceptance publishes/selects semantic files, advances the checkpoint and saves terminal state; these writes are not one crash-safe transaction. Tool/schema/runtime modifications permitted by the skill remain host-managed, not installed or rolled back by semantic acceptance. Direct save, selection, promotion and filesystem operations can bypass the gate. Finalization requires accepted/incomplete state; presentation failure does not undo publication. Complete evidence-based reporting and browser opening remain host responsibilities, separate from terminal tool return (RT-RTE-evolution-admission, RT-RTE-version-override, workflow account).
What the improvement pathway establishes
EvoOntology affords improvement-directed self-change: task evidence can inform revised semantic guidance that later tasks consume. That is a supported architectural contribution, alongside wired history-to-workload reuse. Whether criticism of a consumed explanation improved capacity for future action remains uninspected; no outcome or attribution evidence closes that claim. Conditional performance admission does not accept every semantic proposition or verify the proposed causal mechanism (RT-RTE-evolution-admission, EPI-RTE-gate-aggregation).
The same runtime record assesses the independent properties separately:
- Theory-builder condition 1, localized content: wired. A whole hypothesis and rejection notes can persist; explanatory content remains host-dependent and finer premise decomposition is not enforced.
- Condition 2, content-dependent consumption: claimed. The skill requires intervention decisions to follow attribution; storing nonempty text does not establish such dependence.
- Condition 3, content-directed criticism: claimed. Competing explanations and revised blame after ineffective interventions are requested, not observed.
- Condition 4, iteration: claimed. Rejected explanations should shape later candidates; durable rounds permit but do not demonstrate reuse.
- Reflection: claimed. The method requests representations of tool/schema/runtime limitations and interventions back into them; the causal directions are not observed. Reflective theory building remains uninspected.
- Autonomous theory building: claimed. Internal search is assigned to computation within user-authorized scope, without role-by-role evidence of exercised compliance. Reminder wiring does not establish autonomous launch.
- Self-improvement: afforded. At the ontology-plus-shipped-skills/tools boundary, over later tasks/sessions, the objective is reproducibly better analysis without unacceptable regression. Evidence-directed updating and subsequent guidance are designed, but later operation causally dependent on the update is unestablished. This is a dormant pathway, not exercised or successful self-improvement.
For requested semantic reuse, the assessment is therefore a wired guidance service with consumer-specific fidelity and trust limits. For reproducible ontology improvement, it is host-directed experimentation with an API-local recorded-input veto, not a trustworthy all-path improvement guarantee. For automatic evolution, inspected reminders establish prompting opportunity rather than initiation. These scenario-relative conclusions would change with linked build/evolution traces showing grounding and criticism-to-next-round dependence, authenticated paired reruns under fixed conditions, and subsequent tasks demonstrating effects of the retained change. Component attribution would additionally require independent variation of the alleged mechanism. Enforced immutable versions, observation provenance, partition isolation and all-path admission would strengthen guarantees that currently depend on host obligations (RT-RTE-evolution-admission, RT-RTE-version-override, MEM-RTE-reminder-delivery).
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No observed execution or causal comparison | SRC-1; RT-RTE-build-publication, RT-RTE-evolution-admission, RT-RTE-benchmark-evaluation | Frozen implementation/instructions and attributed README reports; external data/services excluded | Reproduced grounding, successful task delivery, improved capacity, exercised or successful self-improvement | Linked execution artifacts, controlled reruns and later-consumer outcomes with attribution evidence |
| InsightBench missing-part conflict, retained as report-verification limit | SRC-1; RT-RTE-benchmark-tool-loop, MEM-RTE-insight-guidance, MEM-RTE-trace-retention, MEM-OBJ-trajectory; report limit, profile verification | Selective tool-loop/normalized-trace inventory; verifier inspected benchmarks/insightbench/insightbench/agents.py |
Complete InsightBench recovery/history/context or memory-axis enumeration; all execution using ToolCallingChat; fallback retaining semantic-tool access; normalized trajectories exhausting analytical history | Canonical fallback and checkpoint/history part declarations with producer, persistence and later-consumer traces; runs for exercised recovery |
| Fallback/history qualifications cannot be reduced to semantic absence | SRC-1; MEM-RTE-insight-guidance, RT-RTE-benchmark-tool-loop, report limit | Verified RuntimeError fallback to baseline generate_code with semantic_manifest=None; checkpoint reload of insights_history/semantic_events and history consumption | Absence of semantic influence in fallback, successful fallback or exercised checkpoint reuse; enriched schema can already carry semantic content | Full fallback input lineage and execution traces showing checkpoint use and downstream behavior |
| Semantic grounding exceeds structural checks | SRC-1; MEM-OBJ-evidence, EPI-RTE-version-validation, EPI-CLM-grounded-commitment | Root validation and host Build policy | Automated truth certification, faithful observed grounding or accepted novel claims | Reproducible probes, actual object inputs/outputs and evidence-consuming semantic acceptance decisions |
| Gate inputs and evaluator separation are host-dependent | SRC-1; MEM-OBJ-evaluation, EPI-RTE-gate-aggregation, RT-RTE-benchmark-evaluation | Recorded-input aggregation, adapters and instructed data partitions | Authentic independent reproducible improvement, evaluator isolation or verified causal explanation | Provenanced paired artifacts, repeated trials, fixed conditions and audited separation; component interventions for mechanism claims |
| Administrative bypass, mutable version names and nontransactional publication | SRC-1; RT-RTE-version-override, RT-RTE-evolution-admission | Root store, selector and session methods | All-path gate enforcement, immutable replay identity, crash-safe acceptance or guaranteed rollback of host code changes | Enforced admission/permissions on every write path, content-addressed versions and transactional/recovery evidence |
| Lossy traces and unreliable BIRD version attribution | SRC-1; MEM-RTE-trace-retention, MEM-OBJ-trajectory, RT-RTE-task-execution | Project events and benchmark recording projections | Complete replay, authenticated manual observations or trustworthy version-conditioned diagnosis from labels alone | Full linked raw observations and actual loaded-version digests retained by each recorder |
| Consumer-specific projection and binding uncertainty | SRC-1; EPI-RTE-semantic-projection, MEM-OBJ-constraint, MEM-RTE-bird-guidance, MEM-RTE-ddr-guidance, MEM-RTE-insight-guidance | Root and local retrieval; binders/shortcut enforcement incompletely inspected | Universal withdrawal, all-constraint/evidence delivery, accurate physical grounding or universal payload budget | Full binder/shortcut traces and projection contracts, lifecycle preservation at every reader and byte/token enforcement |
| Downstream evaluator licenses are incompletely assessed | SRC-1; EPI-RTE-sql-comparison, RT-RTE-benchmark-evaluation | BIRD scorer inspected; detailed DDR/InsightBench metrics unassessed | System-complete semantic-checking verdict, SQL intent/transfer proof or generalized score warrant | Evaluator inputs, prompts, reductions, reference provenance and domain-validation evidence |
| Metadata does not authenticate sources or freeze batches | SRC-1; MEM-OBJ-source-references, MEM-OBJ-checkpoint, MEM-RTE-reminder-delivery | Pointer storage and latest-trajectory checkpoint advancement | Valid referenced evidence, exact frozen-batch membership or chronological separation from metadata alone | Verified referenced artifacts, immutable batch manifests and checkpoint-to-batch checks |
| External control, model identity and context behavior remain uninspected | SRC-1; RT-CMP-host-model, RT-CMP-bird-provider, RT-CMP-ddr-provider, RT-CMP-tool-calling-chat, RT-CMP-llm-judge, RT-RTE-benchmark-tool-loop | Shipped wrappers/instructions, excluding live hosts/providers | Immutable model continuity, provider-side learning/retention claims, universal sandbox or host context/approval guarantees | Pinned model artifacts/service contracts, host configuration and context/permission traces |
| Generic admission control and opaque payload origins remain unresolved | SRC-1; RT-RTE-version-override, MEM-OBJ-evaluation, MEM-OBJ-source-references, MEM-OBJ-checkpoint; profile verification | Generic supplied-content APIs, original thresholds and reference bodies | Complete manual/automatic control, original lineage, payload form/authority or trace-source classification | Per-content decision provenance and inspected actual payload/update/consumer chains |
| Terminal return is not complete visible delivery | SRC-1; RT-RTE-evolution-admission, MEM-RTE-explorer-projection, workflow account | Terminal-state finalization and HTML/preview generation | Complete host reporting, actual panel opening or human review | Saved final reports and host/browser delivery traces |