h t t p s : / / a r x i v . o r g / a b s / 2 6 0 7 . 1 2 9 6 2

Ingest: Form, Not Content? Placebo-Controlled Self-Repair

Type: kb/sources/types/ingest-report.md

Classification

Scientific preprint by Mehmet İşcan, affiliated with PythaLab at Yıldız Technical University, dated 14 July 2026. The paper presents PoPE, Popperian Placebo-controlled Evaluation, with reported preregistrations, executable audits, experiments, and a program ledger. Its contribution is an evaluation methodology and a bounded negative characterization, not a demonstrated successful repair controller. The experimental materials are available only on request; this ingest assesses the paper, without inspecting or executing its implementation.

Summary

PoPE treats generated programs as conjectures and execution failures as oracle-relative refutations, then asks whether learned error content improves retries beyond the effects of its delivery form and sampling budget. On a selected 40-unit resistant band with a 1.5B code model, a fixed prompt scaffold and four generations per arm and unit, the live error-conditioned prompt unlocked 10 units on public tests versus 12 for its content-ablated placebo. In a separate weight-channel comparison, the genuine-error adapter and unadapted base each unlocked 8 units, versus 10 for a deranged-error adapter trained under the same prescription. Neither channel confirmed content-attributable superiority; hidden-test confirmation was not run, and equivalence was not tested. The paper also preserves and withdraws an apparent allocation gain after a draw-budget audit. Its interpretation distinguishes conditioning a proposer with past criticism from independently testing each new proposal. The controls make this distinction useful for evaluation, but the nulls do not establish its interpretation as the cause of failed transfer.

Quotes

No source quotes have been retained yet.

Connections Found

For the occasion, PoPE is a concrete evaluation of a Popper-inspired repair mechanism and a methodological example for an experiment identifies only the contrast it actually runs. Its content controls and withdrawn allocator positive show why a comparison against an inactive baseline alone cannot identify the contribution of error content or adaptive allocation. The evidence remains specific to the chosen prompt scaffold, adapter configuration, resistant population, and public-test screen; scaffold matching does not mean exact token matching.

The paper also compares with oracle accumulation improves the selection environment. Both distinguish failure information used before generation from checks applied to subsequent candidates. PoPE does not test accumulation of new checks or improvements in their discrimination. Its retained supersession records illustrate persistence of revised research judgments, a different object from persistent improvements in the generator.

Learning Claims (our opinion)

The prompt mechanism learns error severity and routes among six repair actions using an error ontology, a Hedge reservoir, LinUCB, and an offline-fitted interaction matrix. It changes prompts and controller state while leaving generator weights frozen. The weight mechanism trains a QLoRA adapter on 1,964 genuine task–failing-code–error pairs from 236 non-evaluation units. Its placebo uses a complete derangement of the error blocks while preserving the training prescription and error-family marginals. These are different adaptation operations tested on the same resistant band, not two instances of a single parameter update.

The Popperian commitment is explicit: failure under an external execution oracle can refute a program relative to that oracle, while passing remains provisional. The ontology, six actions, fitted severity, low-rank adapter, promotion thresholds, and public/hidden test division are implementation choices. They do not follow uniquely from that epistemic commitment. In Commonplace's theory-refinement terms, a failed program can supply an addressable candidate for repair. Controller fitting and adapter fitting do not by themselves demonstrate revision of an explicit tentative theory. The paper does not establish a recurrent system that retains, revises, and successfully reuses such theories across tasks.

The effective update spaces also bound the result, as in learning inside a fixed decomposition. Prompt adaptation operates through the supplied error representation and action scheme; weight adaptation operates through one adapter configuration and training-target structure. Neither comparison tests alternative decompositions. Failure to confirm benefit can therefore motivate changing representation, data diversity, or model capacity, but cannot identify which caused the null.

The strongest lesson for our account is to separate learning from criticism from the continuing availability of criticism. Error-derived state may influence a proposal without testing that proposal. The source's further interpretation that compiled criticism loses its external epistemic role is a distinction about where the information operates; it is not an experimentally isolated explanation of the nulls. The persistent withdrawal of the allocator claim demonstrates the reported program's correction discipline, not autonomous scientific discovery or a measured long-term learning advantage.

Extractable Value

  • [quick-win] Two comparisons for a semantic-benefit claim. Require the learned treatment to exceed both the intervention-free baseline and channel-specific content controls before crediting its content. PoPE implements this with preregistered effect thresholds plus paired statistical diagnostics. Reuse the control logic, while designing controls for the actual representation: its scaffold and training-data controls leave some surface statistics unmatched.
  • [quick-win] Persistence should retain invalidated positives. The paper keeps the superseded allocator result, its accompanying near-null adaptive–random control, and the corrected matched-budget result connected by a supersession record. This supplies a concrete example for retaining which claim was withdrawn and why, rather than leaving only the latest verdict.
  • [just-a-reference] A bounded negative result for Popper-inspired repair. Neither learned prompt content nor genuine-error adapter training confirmed superiority under the 40-unit public-screen setup, four outputs per arm and unit, and fixed scaffold/adapter choices. This constrains claims about this implementation; it neither refutes Popperian learning nor establishes that placebo content is superior.
  • [experiment] Test proposal guidance separately from retained testing. For a learning paradigm built around conjecture and criticism, compare failure-derived guidance with and without checks reapplied to new candidates. This is our proposed transfer from the paper's operational distinction, not an experiment PoPE reports or evidence that accumulated checks already improve Commonplace.

Limitations (our opinion)

The headline experiments concern a selected resistant band, Python tasks, small models, one prompt scaffold, and one QLoRA configuration. A zero-pass history in a finite candidate pool does not show that correct programs are absent from the generator's distribution. The low-powered comparisons do not establish equivalence, non-inferiority, or general impossibility. Hidden-tier confirmation was deferred after the negative public screen, so these results do not measure the stated public-and-hidden success endpoint.

The controls preserve declared scaffold components or training properties, not exact lexical distributions and token counts. Only output-generation counts are matched; input tokens, training cost, FLOPs, and elapsed time are not. Adapter capacity, training-target structure, and task diversity are not separately identified. These residual differences limit both semantic attribution and efficiency claims.

The paper's confirmed unit-by-portfolio interaction in public-test progress is not controller superiority. Its two-point generator-scale observation comes from cycle logs and is descriptive, not a scaling law. The allocator's earlier positive is explicitly superseded. Combining these distinct records into one overall learning gain would erase their different endpoints and evidential status.

Preregistration, audits, code, and frozen manifests are described but have no public deposit. This prevents independent verification of the claimed precommitments and execution from the retained paper alone. A tracked failure in an earlier HEF audit is disclosed; the author states that it does not affect the headline prompt and weight stacks. The reported audits reduce some gross-failure explanations but do not constitute a reproduction. Finally, conditioning, testing, and persistence are usefully distinguished, but the experiments do not isolate the author's philosophical explanation against simpler accounts such as insufficient model capability or inadequate training diversity.

Add PoPE's withdrawn allocation positive as a bounded evidence example in an experiment identifies only the contrast it actually runs, preserving the superseded and governing comparisons so the Popperian-learning assessment has a concrete case of criticism revising a retained research claim.