Improving an agentic system crosses the natural-language/symbolic boundary

Type: kb/types/note.md · Tags: foundations, constraining, self-improving-systems

In an agentic system — one whose operation runs through a model interpreting retained artifacts — the changes that improve reliability characteristically move behavior across the boundary between natural-language and symbolic form rather than staying within either. A self-representation covering only one form can therefore represent the side such a change starts from but not the side it must land on, and the change escapes governance at exactly the point where the improvement happens.

The asymmetry sorts behavior between the forms

The engine is an error profile, since scheduler-LLM separation exploits an error-correction asymmetry: work that can be fully specified — counting, membership, state tracking, scoping — is exact on a symbolic substrate and error-prone inside a model, while judgment that cannot be fully specified has nowhere to live but natural-language content. A system under reliability pressure is therefore continuously sorting. New behavior enters as natural-language instruction, because that form is where a half-understood constraint can first be stated; as operation clarifies which part of it is bookkeeping, that part is pushed into code. The movement is codification, the pressure behind it is the constraining gradient from convention to enforcement, and the sorting is never finished, because codifying a rule requires an oracle for it and the boundary of automation is the boundary of verification.

The improving changes are crossings

The sorting runs in both directions, and each direction is a boundary crossing:

Natural-language to symbolic. A convention that decays into violations becomes a validator; a membership claim becomes an enforced mark. The improvement is precisely the acquisition of a formal consumer — the unenforced natural-language version of a completeness claim is a trap, since stale indexes are worse than no indexes: it tells an exhaustive consumer to stop looking while members are missing. What made the system more reliable was not better natural-language but the crossing itself.

The Commonplace reference case records one observed crossing in which a natural-language completeness rule acquired formal consumers.

Symbolic to natural-language. An enforcement cannot be repaired from inside itself, because the standard it should have applied is not in the code — the code is the thing under suspicion. The correction has to land in natural-language content. That happens two ways: the enforcement misfires, rejecting what it should accept or encoding a decision the system has outgrown, and the rule must be re-derived from revised intent; or the enforcement runs correctly and its output shows that the natural-language instruction was wrong. Enforcement whose rationale is not revisable drifts opaque; natural-language guidance whose enforcement is not updatable decays into advice.

The same reference case records the reverse crossing: symbolic execution exposed a faulty natural-language search recipe, which was then revised.

Changes confined to one form exist, and some of them matter: a code-only bug fix under an unchanged natural-language contract genuinely improves reliability. The claim is not that single-form changes are worthless but that they cannot move the line — what the system leaves to interpretation and what it enforces. Moving that line is what a crossing is, and it is where the reliability gains that survive a change of operator concentrate.

Consequence for reflective coverage

Since reflective coverage is graded across representational forms, the question is which grades suffice, and for agentic systems the answer follows from where the improving changes live. Natural-language-only coverage lets the system revise what it recommends but not what it enforces, so accepted recommendations accumulate as advice. Symbolic-only coverage lets it adjust what it enforces but not why, so enforcement loses its path back to intent. Either way the improving change lands outside the self-representation, so it reaches later behavior without passing through anything the system represents about itself — and nothing in the loop governs it. Governed self-extension in an agentic system therefore needs modification-grade coverage of both forms, and of the mappings between them.

Scope

  • The claim characterizes where reliability gains concentrate under the stated pressure; it is not a universal over every improvement. A system without reliability pressure, or with no formal consumers at all, is outside the argument.
  • It concerns the natural-language/symbolic pair. The distributed-parametric form sits under selection-grade levers in current agent systems, and this argument does not extend to it.
  • The mechanism carries the claim; the direct evidence base is the single repository trace used in both directions above, set out in full in Commonplace as a reflective system. One trace is not a sample; classifying external agent systems by which forms their improvements touch is the test beyond this repository. A first such classification: Knowledge-Centric Self-Improvement is a natural-language-only improvement loop — accepted claims accumulate as advice to fresh agents, as natural-language-only coverage predicts — that succeeds because its symbolic half, benchmark verification, is supplied externally, fixed, and already adequate, so the enforcement line never needs to move; the settings its authors leave untested are those where it would.

Relevant Notes: