LLM reliability

Type: types/tag-readme.md

How LLM outputs depart from intended requirements, and how to prevent, detect or correct those departures. Assign this tag to substantive accounts of specification gaps, interpreter conformance, execution variation or their remedies, including oracles and architectural separation. Evaluation covers what a check establishes; this tag requires a connection to deviation or correction. Diagnosis must name the governing reference.

The Taxonomy

• LLM output deviation requires three-way diagnosis — the synthesis: the three diagnostic questions, why prompt narrowing, error correction, and sampling control target different relations and so cannot completely substitute for one another, and how to tell the three apart empirically
• Prompting changes failure diagnosis with the semantic reference — distinguishes package conformance, the adjudicated English requirement and behavioral prediction
• Clear instructions can compose into an unsettled contract — the JSON/exception case exposes uncertainty in which obligations apply before diagnosing interpreter failure
• out-of-spec-output-is-a-failure-of-the-interpreter-not-the-spec — source 2, interpreter failure: the output falls outside what the spec allows; a property of the interpreter, with the worked failure catalogue
• traditional-software-can-bracket-executor-conformance-llm-systems — scopes the executor-conformance comparison to ordinary English instruction execution
• execution-indeterminism-is-a-property-of-the-sampling-process — source 3: the same prompt gives different outputs across runs; a property of the sampling process, theoretically eliminable
• LLM debugging separates specification gaps, instruction violations, and run-to-run variation — three checks for choosing a debugging move; failure frequency alone cannot identify the defect

Error Correction Theory

• error-correction-works-above-chance-oracles-with-decorrelated-checks — the core theory: error correction is viable when oracles have discriminative power (TPR > FPR) and checks are decorrelated; amplification cost scales with 1/(TPR-FPR)²
• systematic-prompt-variation-serves-verification-and-diagnosis-not-explanatory-reach-testing — controlled framing changes do two different jobs here: decorrelate weak checks for verification and expose brittleness under semantically fixed prompts; distinct from Deutsch's explanatory-reach test

Oracle Theory

• oracle-strength-spectrum — oracle strength as a gradient from hard (deterministic) to no oracle (vibes); the engineering move is to harden oracles progressively
• reliability-dimensions-map-to-oracle-hardening-stages — Rabanser et al.'s four reliability dimensions each target a different oracle question; each can be hardened independently
• the-augmentation-automation-boundary-is-discrimination-not-accuracy — crossing from augmentation to automation requires per-instance discrimination, which is empirically stagnant; external oracle construction is the practical path
• knowledge-storage-does-not-imply-contextual-activation — relevant knowledge can be present but remain unelicited; activation failure appears when probe retrievability is high but spontaneous emergence is low
• elicitation-requires-maintained-question-generation-systems — strategies for closing the activation gap, ordered by expertise required; composes probes into maintained review architectures
• the-boundary-of-automation-is-the-boundary-of-verification — synthesis: three independent lines of evidence (oracle theory, labor economics, frontier-lab predictions) converge on verification cost as the structural determinant of automation
• evaluation automation is phase-gated by comprehension — phase model inside evaluation loops: automation only generalizes after manual comprehension and calibrated specification produce discriminative judges

Aggregation & Correction

• synthesis-is-not-error-correction — merging agent outputs propagates errors; voting discards minorities and corrects them; the aggregation operation must match the decomposition structure

Architectural Responses

• scheduler-llm-separation-exploits-an-error-correction-asymmetry — separation works because bookkeeping admits cheap error correction (hard oracles) while semantic work resists it; mixing forces bookkeeping onto the expensive substrate (also computational-model)
• specification-level-separation-recovers-scoping-before-it-recovers-error-correction — OpenProse-like DSLs recover frame isolation before gaining hard-oracle bookkeeping; an intermediate regime (also computational-model)

Related notes in other areas

• enforcement-without-structured-recovery-is-incomplete (learning-theory, constraining) — the enforcement gradient covers detection and blocking but not recovery; oracle strength constrains viable recovery strategies
• semantic-review-catches-content-errors-that-structural-validation-cannot (kb-maintenance) — four semantic checks that are decorrelated weak oracles for content errors
• spec-mining-as-codification (learning-theory) — the manufacturing step: extracting deterministic checks from observed behavior to construct oracles
• silent disambiguation is the semantic analogue of tool fallback (observability, computational-model) — adjacent distinction: some bad outcomes come from hidden semantic recovery after an ambiguous spec, not from interpreter failure inside a clear spec

Sources

• Ma et al. (Sep 2025) — Prompt Stability in Code LLMs — empirical evidence: separates all three phenomena methodologically; performance-stability decoupling confirms they are independent

Related Tags

• learning-theory — the parent: oracle and verification theory applied here to LLM output deviations
• computational-model — the scheduling architecture that separation notes describe; error correction explains why it works

Other tagged notes

• A bare writing prompt does not determine its intended contribution - Separates the contribution a bare writing prompt leaves underdetermined from empirical claims about how experts and LLMs supply the missing purpose.
• A goal-holding interpreter fails soft, and its workarounds tax a bounded budget - A procedure compiles its goal away, so a blocked step fails loud and hard; an interpreter holds the goal and re-routes, so failures are absorbed as a per-encounter tax on bounded capacity — silent, accumulating, and softly saturating
• A guard against a conjectured requirement removes the experiment that would test it - A guard against an unstated requirement holds the condition at the value where the conjecture and its negation agree, so it immunizes the conjecture; explains why test-selected agents guard by default and when omitting is the cheaper experiment
• Agent orchestration needs coordination guarantees, not just coordination channels - Coordination channels say how bounded contexts interact, but the missing discriminator is which guarantee prevents contamination, inconsistency, amplification, or liability diffusion across the composed system
• Brainstorming: how to test whether pairwise comparison can harden soft oracles - Staged test plan for whether pairwise comparison improves soft-oracle properties (discrimination, stability, calibration) in LLM evaluation loops
• Brainstorming: maintainability oracles for agentic development - Explores candidate signals, calibration experiments, authority levels, and workflow placements for evaluating maintainability in agent-generated code
• Context contamination operates below an agent's compliance reasoning - A controlled test found fine-grained stance drift despite explicit detection and refusal; exclusion guarantees non-exposure, while instruction-level mitigation remains an empirical question
• Error messages that teach are a constraining technique - In agent systems the error channel is an instruction channel — making errors teach the fix is nearly free and eliminates the agent's need to diagnose, an orthogonal axis to enforcement strength
• Generation confidence does not by itself certify soundness - Distinguishes next-token probability from factual truth and inferential validity: confidence can support correctness decisions only after task-specific validation, and high-assurance acceptance still needs a separate check
• LLM generation can hide a relaxed goal where human writing exposes a stall - An LLM can ship fluent output after silently relaxing an unmet goal, while human composition may expose the same gap as a stall; a conjectural mechanism for why readers inherit the check
• LLM↔code boundaries are natural checkpoints - LLM↔code boundaries expose concrete inputs and outputs for inspection and replay; deterministic execution preserves rather than corrects a wrongly interpreted argument
• Memory-backed personalization can look like model improvement - Distinguishes user-specific gains supplied by retained intent from gains in the model that interprets the assembled context.
• Natural-language meaning is coordination among learned approximations, not approximation of a target - Untested hypothesis extending Lampinen's learned-approximation mechanism to natural language: no external semantics exists to approximate, so meaning is coordination held by weak checks; recasts constraining and deviation diagnosis
• Operational signals that a component is a relaxing candidate - Operational signals for when a component likely encodes a brittle proxy theory rather than an exact specification and should be relaxed instead of codified harder
• Programming practices for prompts need evidence matched to the question - Maps typing, testing, code extraction and versioning to the questions they answer; distinguishes requirement adequacy, conformance, predictive fidelity and execution variation.
• Reasoning production is not reasoning evaluation - Review and critique systems need independent process-validity checks because a model can substitute answer reconstruction for reasoning evaluation
• Review automation should target verifiable subroles before reviewer identity - Scholarly-review automation should decompose reviewer work into separately verifiable subroles before giving an AI system reviewer-level authority
• Topology, isolation, and verification form a causal chain for reliable agent scaling - Topology, isolation, and verification may form a strict dependency chain rather than independent design choices — tested against the simpler account that good decomposition implies the other two
• Weakly discriminated qualities tend to be underselected - Statistical conjecture: under named proposal-selection conditions, unequal oracle discrimination yields unequal enrichment; absolute degradation needs an additional directional mechanism