Operational signals that a component is a relaxing candidate

Type: kb/types/note.md · Status: seedling · Tags: learning-theory

The fixed-artifact distinction says some codified components are exact-spec artifacts while others are proxy theories. That note offers confidence signals, but running systems need more operational tests for detecting proxy-theory components before full composition failure. The signals below shift confidence — they detect badly-fitting theories, not the theory/spec distinction itself, since a well-regularised learned component can pass all five early signals yet still encode a proxy theory.

Signals that a component encodes theory, not specification

Brittleness under paraphrase or reordering. If the component breaks when inputs are rephrased, reordered, or padded with irrelevant content, it's relying on surface patterns rather than capturing the underlying structure. Metamorphic tests can detect this systematically. Rabanser et al.'s agent reliability study operationalises a related signal as R_prompt — prompt robustness — and finds it is the key differentiator among robustness sub-dimensions: models handle genuine faults gracefully yet remain vulnerable to surface-level instruction rephrasings. Ma et al.'s PromptSE framework provides a richer operationalisation: psychologically grounded emotion/personality templates at three calibrated perturbation distances (d=0.1 light lexical, d=0.2 moderate style, d=0.3 substantial transformation) across 14,760 variants and 14 models. Their finding that smaller models can achieve superior stability (Qwen-1.5B highest AUC-E) supports the interpretation that brittleness detects badly-fitting theories rather than capacity limitations. The Mazur position-bias benchmark extends the brittleness signal to the LLM-as-judge layer: across 27 judges on 193 sibling-edit story pairs, the median judge flips its pairwise winner in 44.8% of decisive cases when display order alone is swapped — the signal measured on the evaluator, not the task-solver. None of these sources connect the brittleness signal to the fixed-artifact distinction, but we interpret the findings as evidence that paraphrase brittleness is a measurable and prevalent signal — exactly what you'd expect if many agent components encode theories about input format rather than specifications of meaning.

Isolation-vs-integration gap. The component performs well on unit tests but fails in integration. This was exactly the vision-features pattern: each feature (edge detection, corner detection) worked in isolation, but they didn't compose into "seeing." A growing gap between unit and integration performance is a relaxing signal. Note that composition failure has multiple causes — interface mismatches, protocol errors, environmental dependencies — so this signal is not conclusive on its own. It's strongest when the component's unit tests pass cleanly (ruling out implementation bugs) yet integration still degrades.

Process constraints rather than outcome constraints. The component encodes "always do steps A, B, C" rather than "output must satisfy property X." Process constraints are theories about how to achieve an outcome; outcome constraints are closer to specifications. The more process-heavy, the more likely it's a relaxing candidate.

Hard to specify failure conditions. If you can't articulate what a failure of this component would look like before seeing one, the spec is probably a theory. Arithmetic failures are obvious (wrong number); vision-feature failures are only obvious in retrospect.

High sensitivity to distribution shift. If the component works on training/development data but degrades on slightly different inputs, it has overfit to a particular theory of what inputs look like.

Composition failure (late-stage). The five signals above are early-detection — you can test for them before the system fails in production. Composition failure is the late-stage confirmation: individually sound components that don't compose into the target capability are the strongest evidence that their specs are theories, not definitions. This was the vision researchers' experience — each feature worked alone, but the pieces didn't add up to "seeing." The distinction matters because composition failure is expensive to discover (you need to build and integrate) while the early signals are cheaper to test.

Using the signals

These aren't binary tests. They're indicators that shift your confidence about where a component sits on the oracle strength spectrum.

When signals fire: - Keep the component in a replaceable slot — clean interface, swappable implementation. - Invest in integration tests over unit tests for this component. - Maintain alternative candidates (different decompositions, learned replacements) so relaxing is cheap when the time comes.

When signals don't fire: - The component is likely closer to exact-spec territory. Codify harder — spec mining provides the operational mechanism for extracting deterministic verifiers from observed behavior. More tests, stricter contracts, deterministic implementation where possible.

Open questions

  • Can these signals be measured automatically, or do they require human judgment? Brittleness and distribution sensitivity seem measurable; "hard to specify failure" seems inherently subjective.
  • Is there a feedback loop? Investing in integration tests for suspected vision-features might itself reveal whether the theory is wrong, accelerating the relaxing decision.

Relevant Notes: