Warranted autonomy is bounded by oracle domain

Type: kb/types/note.md · Tags: foundations, self-improving-systems, evaluation

Autonomy — how much of an improvement pathway runs without a person — is a separate gradient over a self-improving system. This note concerns the pathway shape that has a gate: the proposal-selection improvement loop, where candidates are evaluated and can fail to be adopted. Bare autonomy is free: hand the gate to a model with a rubric and no human runs it. More autonomy strengthens that attribution, but it is not automatically a better design. What is bounded is warranted autonomy — a loop that runs unattended and can still be trusted with what it accepts.

Warranted evaluation autonomy extends only to the candidates the available oracle can assess with the required confidence, since the boundary of automation is the boundary of verification. Outside that domain, unattended evaluation remains possible but unwarranted.

Strength, strictness, and domain

Oracle strength concerns how reliably a check discriminates relative to the objective and what its result establishes. Strictness concerns how much evidence the gate demands before accepting. Domain is the range of candidates the oracle's strength, at whatever strictness is set, suffices to warrant — distinct from a claim's own explanatory reach and from how far search can explore, two other senses of "reach" this cluster uses for different things. Increasing strictness under a fixed oracle usually narrows acceptance: uncertain candidates are rejected or deferred so the remaining acceptances carry more assurance.

Strengthening discrimination can instead expand the warranted domain. A better verifier may reject bad candidates the old one passed while also accepting good candidates the old one could not distinguish from bad ones. Test suites, proofs, model judges, and human judgment therefore do not form nested acceptance sets; they establish different things over overlapping domains.

The Gödel machine is the limiting case of assurance obtained through strict acceptance: a rewrite runs only when the machine proves that switching helps under its formalization. It gains warranted total autonomy within that proof surface and excludes improvements it cannot prove. The cost follows from its proof requirement, not from a general law that stronger oracles always accept less.

Commonplace composes several oracles: tests and validators for structural constraints, human judgment for criteria that are not adequately automated. It could hand the latter gates to a model tomorrow and become more autonomous without becoming more warranted. Expanding warranted autonomy requires improving the relevant oracle's discrimination or narrowing the gate to cases it can establish.

The useful questions are therefore both how autonomous is the loop? and which of that autonomy is warranted by its oracles?

Oracle hardening moves the boundary

Oracle hardening can move a gate from unwarranted to warranted autonomy: a rubric becomes a validator, a heuristic becomes a test, or an unmeasured property becomes observable. The oracle-strength spectrum describes different verification surfaces, not a total order over their acceptance rates.

Hardening may expand the warranted domain by resolving cases the old oracle could not discriminate. Where criteria still outrun the oracle, the system must retain a human evaluator, narrow the unattended gate, or accept unwarranted autonomy.

Scope

  • The claim concerns evaluation autonomy, so it is scoped to proposal-selection pathways — a direct evidence-driven update has no gate to hand over, and its trustworthiness is a question about the objective and the update rule instead. Search has a different failure surface, since false-positive generation faces evaluation before retention.
  • Bare autonomy means nobody is required at the gate. Warranted autonomy additionally claims that the unattended gate is reliable enough for its use.
  • Warrant is objective-, risk-, and threshold-relative. No oracle is warranted for every decision merely because it is mechanical.
  • Several oracles may govern different parts of one acceptance decision; their combined domain need not equal the union of what each accepts independently.

Relevant Notes: