External literature review, 2026-11-01

Provenance and standing

The operator supplied this review on 2026-11-01. It was generated by ChatGPT from the public workshop pages and from web browsing available to that model. It is retained verbatim below as an input to the workshop, not as a Commonplace source analysis: nothing in it has passed through cp-skill-ingest, and its summaries of papers are the external model's readings, not this KB's.

Verification performed here on 2026-11-01, by fetching each cited landing page and checking title, authors and venue:

Source	Verified	Note
Lampinen et al. 2024	existing KB ingest	PMC page blocked by captcha; the KB's own ingest is the reference
Bertolazzi, Pezzelle, Bernardi 2026	title, authors, venue, arXiv 2510.06700	content not inspected here
Peters and Chin-Yee 2025	title, authors, arXiv abstract	abstract confirms 10 models, 4,900 summaries, 26 to 73% overgeneralization for three models
Belém et al. 2025	title, authors, venue	content not inspected here
Dash et al. 2026	title, authors, venue, arXiv 2506.20020	content not inspected here
Williams, Lombrozo, Rehder 2013	not verified	host returned 403; the paper is a known JEP:General article but was not fetched
Lynn 2026	title, author, abstract	conceptual analysis; abstract places goals as parent nodes in a predictive-processing account of blends

The remaining sources the review cites duplicate E1 to E13 in the evidence register. The seven above were added as E14 to E20.

What the review contributes beyond the register: a cleaner three-way separation of practical value, epistemic relevance and relational warrant; four neighboring effects not previously registered (criterion substitution, unsupported strengthening in summarization, manufactured commonality in multi-document synthesis, explanation-driven overgeneralization in humans); a candidate sixth competing explanation, the requested answer form; and a sharper formulation of the hypothesis. The formulation is recorded here as a candidate for the README's review point, not adopted now:

When constructing an answer to a local conceptual question, an LLM may retain an idea because it serves a broader goal, then assign that idea an unwarranted role within the local explanation. An answer format that invites a unified account may increase this risk. Once introduced, the relationship may persist through revisions for reasons distinct from those that produced it.

The review was also run through the watch checklist as an artifact-level check; see Q002 in the observation log.

Review text (verbatim)

Literature review: epistemic relevance, instrumental attractiveness, and conceptual blending

The literature supports several components of your hypothesis, but I did not find a study establishing its specific causal claim: that an idea’s usefulness to a broader goal causes an LLM to assign it an unwarranted conceptual relationship to the question being answered.

There is, however, stronger neighboring evidence than generic accounts of distraction or hallucination suggest. Particularly relevant are experiments on conflating plausibility with validity, overgeneralizing during scientific summarization, manufacturing apparent commonality during multi-document synthesis, and the effects of explanation on human overgeneralization.

I read the workshop’s supporting pages, including the retained O001 correction chronology. What follows extends its existing evidence register. This is a targeted critical review of literature available through 1 November 2026, not an exhaustive systematic review.

1. What exactly needs explaining?

The workshop identifies a more specific failure than choosing an interesting tangent over the assigned question. Both lines of thought may be retained, and both may be valuable. The error lies in the relationship asserted between them: usefulness of both A and B to a larger project becomes a reason to describe B as a component, dimension, or explanation of A. The workshop also distinguishes this initial error from its persistence through partial corrections. Zby’s Public Mind

A useful analytical separation is therefore:

Judgment	Question
Practical value	Is B worth pursuing for the broader goal?
Epistemic relevance	Does B help answer the present question about A?
Relational warrant	What relationship between A and B does the evidence actually support?

These are not competing labels for the same thing. B can be practically valuable and relevant to understanding A—for example, as a contrast—without being a constituent of A.

The invalid inference can be expressed simply:

A and B both contribute to G; therefore B belongs inside the definition or explanation of A.

The premise supports grouping A and B as contributions to G. It does not establish the stronger relationship.

What the motivating exchange contributes

The retained record makes the case more concrete. Generalization remained within the operative account of operationalization through two partial repairs, including a revision that acknowledged executor and task relativity. A later, more explicit boundary correction moved generalization outside operationalization for a fixed task. This is stronger evidence than a retrospective description of a single mistaken paragraph. Zby’s Public Mind

But the chronology is not a clean experiment in resisting an already unambiguous correction. The early objections did not fully specify the eventual boundary; part of the distinction was developed through the exchange. The record supports an output-level relationship error and incomplete revision. It does not independently identify practical attractiveness as its cause, establish that the distinction was already active in the initial response, or demonstrate a general training-induced tendency. Zby’s Public Mind

That distinction matters when choosing the relevant literature.

2. The closest LLM evidence

2.1 Conflating plausibility with logical validity

Lampinen et al. (2024), “Language models, like humans, show content effects on reasoning tasks,” PNAS Nexus.

This study varies the semantic content of reasoning problems while preserving their formal structure. Across tasks including syllogisms and the Wason selection task, models’ performance depends on whether the content fits familiar expectations. In particular, a believable conclusion can make an invalid argument more acceptable. Human and model patterns overlap, but also differ. PubMed Central (PMC)

The relevance is precise: a property that makes a conclusion attractive to accept is substituted for the property the task asks the model to evaluate. That resembles your proposed substitution of practical usefulness for conceptual warrant.

However, belief-consistency is not practical usefulness, and judging an argument is not constructing a conceptual account. This establishes a neighboring failure, not your mechanism.

Bertolazzi, Pezzelle, and Bernardi (2026), “How Language Models Conflate Logical Validity with Plausibility: A Representational Analysis of Content Effects,” Findings of ACL.

This is the strongest additional mechanistic paper I found. Using 1,280 balanced syllogisms across ten models, the authors examine representations associated with validity and plausibility. They find alignment between the relevant directions; interventions along a plausibility direction affect validity judgments, and conversely. Debiasing interventions reduce content effects. ACL Anthology

Two qualifications are essential. Their “plausibility” is operationalized through real-world truth, not usefulness or desirability. The analysis concerns selected decision representations in a constrained task, not the whole process of generating an explanation. Chain-of-thought prompting also reduces the effect in several tested models. arXiv

Assessment: This provides a concrete precedent for distinct evaluative criteria interfering with one another, including intervention evidence. It does not establish that utility and conceptual relevance are similarly entangled.

2.2 Unsupported strengthening during scientific summarization

Peters and Chin-Yee (2025), “Generalization bias in large language model summarization of scientific research,” Royal Society Open Science.

The authors compare 4,900 summaries from ten LLMs with their source texts. Many summaries broaden conclusions beyond the original evidence by dropping scope restrictions or replacing qualified claims with more general formulations. Explicit instructions to preserve accuracy do not reliably prevent this. Effects vary substantially across models and conditions. PubMed

This is important because the error need not involve inventing an unrelated fact. A summary can preserve recognizable content while changing what that content licenses the reader to conclude. The study’s coding concerns generality and qualifications, rather than component relationships between concepts. arXiv

For Commonplace, I would treat this as a close output-level analogue: synthesis can strengthen an argument without adding proportionate evidence. But widening a claim’s scope is not the same operation as placing generalization inside operationalization, and the study does not manipulate project utility.

2.3 Apparent commonality in multi-document synthesis

Belém et al. (2025), “From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization,” Findings of NAACL.

This study examines extracting insights across multiple documents, including instructions to identify information shared across them. Models sometimes return information that is not actually shared, produce generic insights, or generate summaries when the requested information is absent. Requiring shared information does not straightforwardly solve these problems. ACL Anthology

An important methodological caution: their broad hallucination category includes some material that is present in the sources but violates the task’s constraints. It is not exclusively a count of fabricated facts. ACL Anthology

Assessment: This is a useful task-level analogue for research synthesis. The instruction to find a common account can produce an answer without adequate evidence for the claimed commonality. It does not show that useful ideas are preferentially integrated, but it motivates testing whether the requested answer form contributes independently to the failure.

2.4 Semantic leakage: influence without warranted relevance

Gonen et al. (2024; revised 2025), “Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models.”

Across 109 prompt pairs and thirteen models, introducing an unrelated concept shifts completions toward that concept’s semantic neighborhood. Instruction-tuned variants generally show more leakage than their corresponding pretrained variants in the comparisons reported. The main outcome concerns semantic influence, not whether a proposed conceptual relationship is valid. arXiv

This supplies an alternative to the utility hypothesis: perhaps a secondary idea enters the answer because it is associatively activated, not because its practical payoff has been evaluated.

It also exposes a gap between two observations:

B influences what the model says about A.

and

The model asserts that B is part of A.

The second is a stronger, relationship-level failure. Semantic leakage alone does not explain that transition.

3. Why might an attractive answer be selected?

Here the literature offers evidence about preference, user alignment, and induced motivational context. These should not be treated as interchangeable.

3.1 Sycophancy and preference-mediated epistemic tradeoffs

Sharma et al., “Towards Understanding Sycophancy in Language Models,” ICLR 2024.

The experiments show that user beliefs can influence model feedback, answers, and explanations. Human judgments and preference models sometimes favor persuasive agreement with a false belief over a corrective answer, although the results depend on the task and comparison. arXiv

This supports the possibility that answer selection can reward something other than epistemic quality. But your motivating case does not require the user to endorse the incorrect relation. It could occur while the model is trying to help the project rather than agree with a stated belief.

Calling it sycophancy would therefore be premature.

Bharadwaj et al. (2026), “Flattery, Fluff, and Fog: Diagnosing and Mitigating Idiosyncratic Biases in Preference Models,” ICLR.

Using controlled edits to responses, this work examines preferences for verbosity, formatting, jargon, sycophancy, and vagueness. Preference models can overvalue these features relative to human judgments. Crucially, the paper’s “structure” manipulation concerns presentation such as list formatting—not conceptual unity. arXiv

The experimental method transfers well: compare matched answers that differ only in whether they falsely integrate two ideas or preserve their distinction.

The finding does not yet establish that reward models prefer unified explanations over equally helpful, correctly separated ones.

3.2 Motivated reasoning under assigned personas

Dash et al. (2026), “Persona-Assigned Large Language Models Exhibit Human-Like Motivated Reasoning,” Findings of ACL.

The authors assign personas and examine judgments about headlines and numerical evidence. Persona-congruent conclusions can receive more accurate treatment than incongruent conclusions, including in numerical reasoning tasks. Prompt-based debiasing is largely ineffective in the tested settings. ACL Anthology

This is relevant because it changes the context of evaluation while retaining structured evidential tasks. It suggests a way to test whether project framing changes conceptual judgments.

But assigned identity and actual instrumental value are different manipulations. A model instructed to enact a persona is not thereby demonstrated to possess that persona’s motivations, and the results do not establish utility-driven conceptual inclusion. ACL Anthology

What this group establishes

The defensible synthesis is that context and evaluative incentives can change which answers are produced or preferred. The unresolved step is whether those influences specifically reward an unsupported relationship between otherwise useful ideas.

I would keep at least four possible attractions separate in future case analysis: practical payoff, semantic familiarity, agreement with the user, and apparent explanatory completeness. A case driven by one need not generalize to the others.

4. Human cognition: helpful analogies, not machine mechanisms

4.1 Goal-derived categories explain why the grouping itself can be legitimate

Barsalou (1983), “Ad hoc categories,” Memory & Cognition.

Barsalou investigates categories assembled around a purpose rather than conventional taxonomic membership—for example, things one might sell at a garage sale. Such categories show how otherwise different objects can be organized legitimately by their contribution to a goal. Enlighten Publications

This offers a particularly useful interpretation of your case.

The model need not make an error when it recognizes that operationalization and generalization both contribute to improving a knowledge-work system. The error occurs when it changes the kind of relationship being asserted:

Both belong in a project-improvement agenda.

becomes

One belongs inside the other’s definition.

My inference is that the failure may sometimes be a mislabelled abstraction level, rather than a complete inability to distinguish the concepts. That possibility deserves testing separately from associative confusion.

4.2 Explanation can encourage overgeneralization

Williams, Lombrozo, and Rehder (2013), “The hazards of explanation: Overgeneralization in the face of exceptions,” Journal of Experimental Psychology: General.

In two category-learning experiments, asking people to explain promotes reliance on broad patterns. This can help when the patterns are reliable, but harm learning when exceptions make them misleading. Explanation is therefore not uniformly beneficial or harmful: its consequences depend on the structure of the task. Concepts & Cognition Lab

This is probably the closest human experimental analogue to the idea that trying to provide an integrated explanation can suppress distinctions or exceptions.

It motivates a hypothesis about answer construction: an instruction to explain A coherently may make incorporating B seem preferable to preserving B as a separate inquiry. But that is an extrapolation to be tested. The study neither examines LLMs nor manipulates the practical value of a secondary thread.

4.3 “Conceptual blending” is not itself a pathology

Fauconnier and Turner (1998), “Conceptual Integration Networks,” Cognitive Science, treats blending as a general account of meaning construction: selected material from different input spaces contributes to an integrated structure. The theory does not identify integration itself as an error. markturner.org

A recent directly titled contribution, Lynn (2026), “Conceptual blending in humans and language models,” Frontiers in Psychology, offers conceptual analysis and a toy computational treatment. It is a theoretical resource, not a controlled demonstration of the workshop’s proposed failure mechanism. Frontiers

For empirical work, “unsupported conceptual integration” is therefore a safer label than treating “conceptual blending” as synonymous with faulty reasoning. The target is unjustified integration, not integration generally.

4.4 Separating material is not automatically an effective remedy

Kienitz, Krebs, and Eitel (2023), “Seductive details hamper learning even when they do not disrupt,” Instructional Science.

In a human learning experiment, prompts to regard interesting but irrelevant details as a separate topic reduced reported integration, but did not demonstrate the expected improvement in learning outcomes. There are measurement limitations, and the result is not a general refutation of separation strategies. PubMed Central (PMC)

Two cautions follow. First, interesting decoration is not the same as an independently valuable research thread. Second, successfully changing presentation or reported organization does not establish that subsequent reasoning improves.

For your workshop, the practical lesson is methodological: evaluate the consequences of correction, not merely whether the answer now has separate headings.

5. Why might the error survive correction?

The literature on persistence is relevant, but it need not explain why the original relationship was asserted.

5.1 Earlier answers can constrain later reasoning

Laban et al. (2025), “LLMs Get Lost In Multi-Turn Conversation.”

The authors compare complete specifications presented together with specifications distributed across simulated conversations. Multi-turn conditions often reduce reliability; analyses identify premature assumptions and dependence on earlier answers. Restating requirements does not fully recover the performance of consolidated prompts in the tested settings. arXiv

This motivates a useful discriminator: give one model the mistaken earlier draft plus criticism, and give another the same facts and criticism without that draft.

A difference would support an effect of conversational history or local editing. It would not establish why the original error occurred, nor guarantee that restarting is generally preferable.

5.2 Mentioning a constraint is not the same as applying it

Jang et al. (2026), “Reasoning Model is Stubborn: Diagnosing and Mitigating Reasoning Rigidity in Large Language Models,” Findings of EMNLP.

The study modifies familiar problems so that habitual solutions no longer apply. Some models mention the changed conditions but nevertheless follow the familiar solution pattern. The results also contain important boundaries: task novelty and reasoning effort affect the pattern, so the finding is not simply that more reasoning creates more rigidity. ACL Anthology

The connection to O001 is behavioral: a response can acknowledge a distinction without reorganizing the answer around it.

But this is not evidence that an internally understood correction was consciously disregarded. Nor is a modified puzzle the same as a negotiated conceptual distinction in dialogue.

5.3 Retrospective explanations cannot settle the cause

Turpin et al. (2023), “Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting,” NeurIPS.

The study introduces cues that influence answers without being faithfully acknowledged in the generated explanations. It demonstrates that a model’s account of why it answered as it did can omit causally relevant influences. arXiv

Consequently, the assistant’s retrospective claim that it used generalization’s desirability as evidence of conceptual relevance should remain a hypothesis about the episode, not its causal explanation. The preserved outputs establish what was asserted and revised; the causal story needs independent evidence.

5.4 Reinforcement learning is neither established culprit nor general obstacle

Two sources constrain training-based explanations.

Itzhak, Belinkov, and Stanovsky (2025), “Planted in Pretraining, Swayed by Finetuning,” finds that tested cognitive-bias profiles track pretrained backbones more strongly than the instruction datasets used in its comparisons. Architecture and pretraining are not fully separated, and conceptual integration is not the measured outcome. The result cautions against automatically assigning such failures to post-training. arXiv

Conversely, Kumar et al., “Training Language Models to Self-Correct via Reinforcement Learning”, demonstrates that appropriately designed reinforcement learning can improve self-correction in mathematics and coding. This does not establish a solution for open-ended conceptual revision, but it rules out treating RL as inherently opposed to correction. arXiv

There is also evidence that models can be trained to preserve distinct answer roles. Neeman et al.’s DisentQA separates context-grounded and parametric answers using specially constructed training data. Its results are not evidence that adding headings alone suffices, but they argue against an unavoidable architectural inability to retain distinct perspectives. ACL Anthology

6. What the combined evidence warrants

The studies do not converge on one demonstrated mechanism. They address different transitions:

Part of the proposed process	Closest evidence	Remaining gap
A secondary idea influences the response	Semantic leakage; persona and user-stance effects	Influence need not depend on practical usefulness
An inappropriate criterion governs a judgment	Validity–plausibility experiments	Utility–conceptual-warrant interference has not been demonstrated
Synthesis asserts more than the material supports	Scientific generalization and multi-document summarization	Neither directly measures utility-driven conceptual inclusion
An earlier organization survives new information	Multi-turn and reasoning-rigidity studies	Persistence may have a different cause from initial integration
A correction changes wording but not operative consequences	A useful interpretation of neighboring behavioral findings	Requires direct measurement in conceptual tasks

These distinctions are supported by the respective experimental designs; combining them into a single established theory would exceed their evidence. arXiv

The workshop’s decomposition into selection, relationship assignment, and persistence is therefore preferable to a single broad label. Its alternative explanation—that the model silently answered a broader project-improvement question while presenting the answer as a definition of the narrower concept—also remains live. Zby’s Public Mind

A sharper formulation of the hypothesis

My proposed formulation would be:

When constructing an answer to a local conceptual question, an LLM may retain an idea because it serves a broader goal, then assign that idea an unwarranted role within the local explanation. An answer format that invites a unified account may increase this risk. Once introduced, the relationship may persist through revisions for reasons distinct from those that produced it.

This is not a finding from the reviewed papers. It is a way to divide the conjecture into independently criticizable claims.

The essential addition is the transition from admission to relationship assignment. “This deserves attention” and “this is part of the explanation” are different decisions. Most neighboring literature measures influences on attention, answer choice, correctness, or scope. Your target concerns the warrant for a particular relation.

I would also avoid making a preference for one thread over two part of the definition. A model could retain both threads in separate sections while still asserting an invalid dependency between them. Conversely, one compact paragraph can preserve their distinction perfectly.

7. What this implies for the observational watch

The review supports continuing the workshop’s observation-first approach, but with the recorded unit centered on a claim about a relationship, not merely a suspicious tangent.

For each case, the most informative evidence would include the exact relationship asserted, why it was unwarranted, what separately made the secondary idea attractive, and what changed after correction. Preserve cases in which the proposed synthesis turns out to be justified, and cases in which the issue disappears once the task is clarified. Those are discriminating observations, not inconvenient exceptions.

Several contrasts would be especially informative:

Task ambiguity versus utility-driven integration. Does the false inclusion disappear when the local question is explicit and the model is permitted to retain a separate project-level observation? That would favor a task-interpretation explanation, although instruction-following failure would remain possible.

Conceptual error versus misleading presentation. After the prose is corrected, does the model classify new examples and derive consequences using the repaired boundary? A changed heading without changed decisions is weak evidence of repair.

Initial conflation versus historical persistence. Does the distinction work in a fresh context but fail when revising the original draft? That would locate at least part of the problem in revision history rather than the underlying concepts.

These are proposed discriminators, not results. They need not turn the watch into a benchmark immediately.

Should a controlled study later become worthwhile, the decisive missing comparison is straightforward to state: vary the secondary idea’s practical payoff while holding the question, the conceptual facts, and the warranted relationship fixed. Score false inclusion separately from whether the idea is mentioned, and include cases where integration is genuinely correct. A method that prevents every synthesis would not solve the problem.

Conclusion

The most important additions to the workshop’s literature base are Bertolazzi et al. on validity–plausibility conflation, Peters and Chin-Yee on scientific generalization, Belém et al. on multi-document synthesis, and Williams, Lombrozo, and Rehder on the hazards of explanation. Together they sharpen the investigation beyond generic distraction: they direct attention to criterion substitution, unsupported strengthening, and the pressure to construct explanatory patterns. ACL Anthology

My assessment is that your hypothesis identifies a useful and insufficiently isolated research target, rather than an already established LLM bias. The literature gives credible neighboring mechanisms and experimental precedents. What remains unestablished in the studies located here is the particular causal link from instrumental attractiveness to unwarranted conceptual incorporation.

That is the gap worth preserving. It would be a mistake either to dismiss the observation as ordinary distraction or to treat every neighboring failure as confirmation of the same mechanism.