h t t p s : / / w w w . m a n n i n g . c o m / p r e v i e w / s u t s k e v e r s - l i s t / c h a p t e r - 7

Ingest: The pivot to reasoning

Type: kb/sources/types/ingest-report.md

Classification

The chapter constructs a thematic route from representation placement and relational modules to modern test-time reasoning, then offers an epistemic argument about actionable doubt and benchmarks. Author: Richard Heimann is a secondary synthesizer. The chapter is strongest when reporting cited ablations and weakest when implying a causal lineage from heterogeneous research programs to current reasoning models.

Summary

The chapter groups VLAE, relation networks, message-passing neural networks, and relational recurrent memory as attempts to organize information and computation so that global structure, object relations, graph interactions, or memory-to-memory comparison become easier to learn. It then turns to modern language-model brittleness: perturbed planning scripts, the reversal curse, and prompts that improve familiar reasoning forms without guaranteeing recomputation under new constraints. Search, verification, explicit states, and test-time compute are presented as attempts to stabilize reasoning. The final “paper doubts versus living doubts” section argues that skepticism earns force when it changes benchmarks or experiments, using ARC benchmark succession to show that a leaderboard win can be local rather than general.

Quotes

No source quotes have been retained yet.

Connections Found

The source strongly supports reasoning production is not reasoning evaluation: a model may reconstruct a familiar answer path while failing to enforce the changed constraints. Each historical architecture also exemplifies learning within a fixed decomposition, because it preselects where information lives and which interactions are expressible. The prompting discussion limits prompt ablation, and ARC turnover compares with known-target benchmarks showing reachability rather than closure. The reflective-self-improvement fit is negative: test-time deliberation changes an answer, not the system's retained organization.

Extractable Value

  1. Reasoning reliability depends on stable invariants, not plausible trajectories -- perturbed river-crossing tasks expose models that adapt the prose around a cached plan without recomputing legal actions. [deep-dive]
  2. Architecture is explicit placement of information and interaction -- VLAE assigns global/local content, relation networks enumerate object pairs, MPNNs define graph messages, and RMCs define interacting memory slots. The resulting benchmark gains are local to those choices. [quick-win]
  3. Prompts are elicitation interfaces, not guaranteed reasoning engines -- explicit states and chain-of-thought can reduce variance in distribution, but novel constraints call for search, executable checks, or external verifiers. [quick-win]
  4. A living doubt changes the rejection surface -- operational skepticism produces a perturbation, benchmark, invariant, or verifier that can make the incumbent fail, rather than merely naming a shortcoming. [deep-dive]
  5. Benchmark succession tests reach better than benchmark saturation -- ARC-AGI-2's collapse after ARC-AGI-1 success reveals overspecialization that the saturated benchmark could no longer discriminate. [experiment]
  6. Test-time compute is not self-improvement by default -- more search during one inference has no durable operative retention and therefore does not meet the KB's self-improvement definition. [quick-win]

Limitations (our opinion)

The chapter's central lineage is thematic, not demonstrated: VLAE compression, relational vision, quantum-chemistry message passing, recurrent memory, chain-of-thought, and contemporary reasoning models solve different problems under different objectives. Benchmark wins show that each supplied decomposition worked locally, but do not isolate a general faculty called reasoning. “Living doubt” is a useful label, yet it risks naming good experimental practice rather than explaining why one benchmark or intervention is discriminating. ARC score comparisons are especially sensitive to model version, compute budget, benchmark exposure, and evolving task design.

Schedule a focused brainstorm on whether “a living doubt changes the rejection surface” adds a distinct mechanism to the existing oracle and proposal-selection notes before promoting it as a new claim.