h t t p s : / / a r x i v . o r g / a b s / 2 6 1 0 . 2 9 1 6 2

Ingest: Why is AI-generated language so hard to understand?

Type: types/ingest-report.md

Classification

This is an empirical arXiv preprint (cs.CL, 21 October 2026, not yet peer reviewed). It combines a corpus comparison with a small controlled fine-tuning intervention, and it states two hypotheses in advance. Author: Thomas Hikaru Clark (NYU), Matias B. Martin and Kyle Mahowald (UT Austin). Clark and Mahowald are psycholinguists in the communicative-efficiency tradition. Mahowald co-authored the 37-language dependency-length study and the 2019 efficiency review that this paper uses as its human premise.

Summary

The paper tests two hypotheses. H1 says that frontier and near-frontier LLMs overuse linguistic features that prior psycholinguistics links to human processing difficulty. H2 says that a post-training pressure for information density pushes LLM output away from human-like language. For H1 the authors collect a new corpus from 14 models (nine frontier or near-frontier, five small open models) under one agent harness: 120 multi-turn chats with a simulated user, 45 agentic bug-fixing sessions on public repositories, and 980 persuasive essays. They count reduced and full object relative clauses, ad hoc denominal verbs ("calendar it"), inanimate subjects with animate verbs ("this asymmetry bites"), stacked noun compounds, length-standardized dependency length, and rare n-grams. They compare each model with two references. The first is genre-matched human text: subtitles for chat, StackExchange for coding, and high-scoring student essays. The second is a round-trip translation of the model's own sentences through Japanese, which holds content roughly fixed while changing wording. Against the round trip, most frontier models overuse rare n-grams, denominal verbs, inanimate subjects and reduced object relatives, and they underuse relatives with "that". Effects are largest in chat and coding and smaller in single-turn essays. Llama models and the Olmo base model change little, and the effect grows from Olmo base to instruct to its reasoning variant. Against human text, the robust excess is in inanimate subjects and rare n-grams. That comparison rests on the student essays, because neither feature has a human baseline in chat or coding. Opus 5.5 scores lower than Opus 5 on 19 of 21 difficulty-linked comparisons. For H2, Qwen2.5-1.5B-Instruct is fine-tuned with GRPO to turn JSON event descriptions into sentences. In this task both members of each syntactic alternation count as correct. Any length penalty then drives a near-categorical preference for the shorter member. The preference carries over to untrained "that" omissions, and to one of four unrelated alternations (the genitive), but only without KL regularization. The intervention result holds within that decomposition: one small model, one task in which the alternation is the only free choice, and a length penalty as the only pressure. It shows that such a pressure is sufficient to produce the shift. It does not show that frontier post-training applied this pressure. The paper does not measure human comprehension of the generated text. The difficulty of each feature is inferred from prior experimental work.

Contribution assessment (our opinion)

The problem matters and is under-studied. Complaints that LLM prose is hard to read are common but anecdotal, and earlier work on LLM style mostly targets word overuse or detection. The paper turns the complaint into falsifiable claims about named constructions with known processing costs, and it adds a mechanism conjecture with a direct, if small, test. Its strongest design choice is the round-trip comparison. It asks whether the model chose a construction over an available paraphrase of the same content, which a genre-mismatched human baseline cannot ask. The gradient within the Olmo family adds weight, because the effect grows with post-training stage on one base. The contribution is mainly descriptive. Corpus evidence covers several model families and settings but few seeds. The human-cost link is borrowed from prior literature. The H2 intervention is a proof of sufficiency in a setting built so that only length separates the options, which makes the trained shift close to predictable from the objective (the paper's own equation 2). The transfer results are the more informative part. The attribution of the corpus pattern to verifiable-reward training is speculation, and the paper says so. On my background knowledge, the specific pairing of psycholinguistic difficulty features with a round-trip paraphrase control is new for LLM output. The broader idea that RL objectives drift language away from human norms has precedents in emergent-communication work, which the paper does not compare. These gaps narrow the causal claim but do not undermine the descriptive finding. The work is a credible first map that needs reader studies and a human-text round-trip control before its central causal story can be treated as tested.

Quotes

No source quotes have been retained yet.

Connections Found

The source is producer-side evidence for the KB's prose-readability work. It measures what LLM writers produce, not what readers pay. Its closest anchor is Dependency locality supports ordering heuristics, not a reading-cost rule. That note's heuristics are warranted for human readers. This paper reports that frontier-model text in chat and coding has longer standardized dependencies than its own round-trip paraphrases, so the heuristics address a real tendency in agent-drafted prose, not only in human drafts. The paper does not test any heuristic. It does not measure comprehension or answer the note's open question about LLM readers. Its background citations, that larger LLMs fit human reading times worse, bear on that question only indirectly. The note also finds that direct reading effects of dependency length on ordinary text are small. A writer who cites this paper as proof of large comprehension costs would overreach that caution.

The paper also gives a measured case for Human-LLM differences are load-bearing for knowledge system design. The note argues the dual-audience problem mostly from first principles. Here, an LLM writer produces constructions that cost a full-attention model little but cost human working memory more. The human cost remains correlational and is inferred from prior studies.

The paper's "compression" is not the opposite of Reverse compression is when LLM output expands without adding information. The two measure different levels. Every feature here is a rate within text the model already chose to write, and the round-trip control holds that content fixed. So the paper measures encoding: how tightly each selected point is worded. It never measures selection: how much content the model includes compared with what a human would write for the same purpose. The walls of text that operators observe in agent output are mostly a selection failure: restatement, hedging, exhaustive coverage. Reverse compression describes that failure. Both findings can hold in one text: too much content, each point packed too tightly. Our conjecture, not the paper's, is that one training pressure could produce this combination. A model rewarded for thoroughness and charged per token keeps the content and squeezes the wording, because wording is the cheap place to save. Either way, "shorter" and "easier to read" are separate targets. It also bounds Two rewrites exposed a syntax-or-repetition tradeoff. Those rewrites found that undoing compressed syntax costs words. The paper's length-penalty result predicts that word-count pressure on an editing pass pushes back toward the constructions the passes were removing.

Two earlier ingests supply its human premise. Large-scale evidence of dependency length minimization in 37 languages finds human dependency lengths below random reorderings, and this paper finds most frontier models above the human level. How efficiency shapes human language is the premise that communicative pressure shapes human language. This paper argues that verifiable-reward post-training replaces that pressure with information density. Finally, AI;DR (AI; Didn't Read) locates reader burden from AI text in excess and unchecked content. This paper locates it in dense syntax and rare word sequences. Naming both keeps a remedy for one failure, cutting length, from being applied to the other.

Learning Claims (our opinion)

The paper's learning mechanism is reinforcement learning on output policy. A reward that pays for task correctness and charges per token shifts the probability of each member of a syntactic alternation toward the shorter one. The shift is near-categorical without a KL anchor to the base model and gradual with one. It also spreads to structurally similar choices the reward never targeted. The authors conjecture that frontier post-training on math, code and reasoning applies a similar density pressure and that the corpus pattern is its residue.

Mapped onto Commonplace concepts, this is weight-level optimization, not theory building. Judged condition by condition:

  • Localized content fails. The changed preference lives in LoRA weights, and no unit states "prefer the shorter variant".
  • Consumption holds only in the weak sense that the trained policy guides generation. No stated theory is consulted.
  • Content-directed criticism fails. The reward scores outputs, and blame reaches parameters by gradient, not by a statement that can be argued with.
  • Iteration holds as optimizer steps that carry updated weights forward.
  • Persistence extends across the run and into deployment of the frozen adapter.

The paper does show learning in the narrow sense of changed future behavior, measured on held-out items. Whether that change counts as improved capacity depends on the objective. It improves reward, and the authors' point is that it degrades human-likeness.

The KB-relevant distinction is the fixed decomposition, as in Learning inside a fixed decomposition inherits its mistakes. The task, the correctness check and the reward fix the outcome space so that both variants are correct and only length differs. The optimizer then improves reward inside that space, and the readability cost of the shorter variant sits outside every signal it sees. The paper's discussion makes the same point when it says correctness is decoupled from humanlike communication. This supports the KB's account. It is a small, concrete case of an objective that omits a distinction the deployer cares about, so optimization drives behavior across it unchecked. It does not test whether adding a communication signal would repair the omission. The improvement from Opus 5 to Opus 5.5 is consistent with such a repair, but the paper has no information on how that change was made. The transfer to untrained alternations is the surprising part. It shows that a pressure aimed at four alternations generalizes along a learned "drop the optional word" dimension, so a narrow objective can change style more broadly than its targets suggest. Far transfer was weak (one alternation of four, without KL only), so the extent of that spread is open.

Extractable Value

  1. Selection and encoding fail separately -- LLM text can include too much content (reverse compression) while wording each included point too densely (this paper). Density saves tokens by choosing reduced relatives, denominal verbs, noun stacks and rare word sequences, which prior work links to human processing cost. The practical consequence is for editing: cutting words while keeping all the content pushes prose toward the dense constructions; the useful cut is deciding what to leave out. This source gives the encoding side a named construction list and a sufficiency mechanism. It holds only on the producer side and correlationally, because reader cost is inferred, not measured on these texts. [quick-win]
  2. A construction-level checklist for agent-drafted prose -- the six measured features (reduced object relatives, ad hoc denominal verbs, inanimate subjects with animate verbs, stacks of three or more nouns, long dependencies, rare n-grams) map closely onto what the KB's clause-packing, mannered-prose and jargon-persistence review lenses look for. A theory note abstracted from this paper could give those lenses an external basis, scoped to features with prior human-difficulty evidence. Each feature's cost in technical KB prose is untested. [experiment]
  3. Word-count pressure on an editing pass predicts denser syntax -- in a task where both forms are correct, a per-token penalty alone produced a categorical preference for the shorter form and spread it to untrained "that" omissions. The result is from one 1.5B model and one task, and it shows sufficiency, not that any frontier model was trained this way. Still, it warns against simplification passes or instructions that set word targets without construction-level guards, the tradeoff the two-rewrites evidence already observed. [quick-win]
  4. Round-trip paraphrase as a control for "did the writer choose this" -- translating a text's sentences through a distant pivot language and back holds content roughly fixed and exposes constructions the writer chose over alternatives. A KB experiment on agent-drafted notes could reuse the design, but it needs a human-text round-trip as a baseline, which the paper lacks. [experiment]
  5. Multi-turn and agentic settings show larger effects than single-turn essays -- the round-trip differences are larger in chat and coding than in essays. The authors speculate that long exchanges demand more compression. KB drafting by agents happens inside long sessions, so this suggests that late-session prose deserves more scrutiny. The suggestion is untested and confounded with genre. [just-a-reference]

Limitations (our opinion)

The paper never measures whether readers understand the generated text less well. It also never measures total length or content volume against a human answer to the same task, so it says nothing about the verbosity readers most often complain of in LLM output; its title should not be read as explaining that. Each feature's cost comes from prior experiments, often on constructed sentences, and the dependency-locality note shows that such costs can be small on ordinary text. The round-trip control has a simpler rival explanation. The authors note that machine translation normalizes output toward frequent forms, which by itself lowers n-gram perplexity, removes rare pairings and favors canonical relatives. No human text was round-tripped, so the size of that baseline shift is unknown. The near-null shift for Llama and Olmo base outputs partly answers this. Those models are weaker writers, though, so their text may already sit near the translator's frequent forms. The human comparisons are uneven. Inanimate subjects and denominal verbs have human baselines only for essays, and the n-gram measures compare with the human reference only for essays. The headline "all nine models exceed human writing" therefore rests mainly on comparisons with grade 6–12 student essays. Those were the best-scored essays, but they are a different register from expert adult prose. Two features come from an external classifier accepting candidates above probability 0.5, and the snapshot reports no validation of that classifier. Significance tests are not corrected for many model × feature × setting comparisons. The seeds are few (10 chat topics, 5 repositories, English only), and the simulated user is itself an LLM. On the mechanism, the H2 setup makes only length distinguish the correct variants, so a categorical shift is close to guaranteed by the objective. The informative transfer results are weak beyond "that" omission. The experiment does not address inanimate subjects, denominal verbs or rare n-grams, which are H1's most robust human-comparison findings. The link to verifiable-reward training at frontier labs is conjecture. The Opus 5 to Opus 5.5 change shows that the pattern varies within one family, and the paper cannot say why. Any KB use should keep "producer-side, correlational, mechanism shown only as sufficiency in a toy task" next to the claim.

Write no theory note yet. The selection-versus-encoding distinction and the thoroughness-plus-token-cost conjecture are first observations, and reader cost is unmeasured. Record the distinction in kb/log.md, and revisit when a KB rewrite shows dense constructions appearing under word-count pressure, which would confirm the paper's mechanism locally. A note would then cite this paper for the construction list and the length-penalty sufficiency result, scoped as producer-side and correlational.

Relevant Notes: