The bitter lesson selects production methods, not representational forms

Type: kb/types/note.md · Tags: learning-theory, deploy-time-learning, foundations

Richard S. Sutton's 2019 essay “The Bitter Lesson” contrasts leveraging human knowledge with leveraging computation through search and learning. That is a claim about production method — how a system's behavior-determining content gets made. The folk compression “structure loses to weights” silently substitutes a claim about representational form — how retained content is encoded and consumed in natural-language, symbolic, or distributed-parametric form. Crossing these two axes yields four idealized quadrants:

hand-crafted learned (search + selection)
distributed-parametric hand-tuned weights and thresholds — half of what the lesson displaced deep learning — the widely cited quadrant
localized forms hand-written feature extractors, then today's prompts, harnesses, curated KBs — the other half, and the lesson's next target prompt, code, and harness search — bounded instances, scaling open

“Localized” groups natural-language and symbolic artifacts; mixed systems should be decomposed into their operative parts. What the lesson originally displaced straddled both rows — hand-tuned weights and hand-written feature extractors — which is itself evidence that the selection ran along the column, not the row. The columns classify each part by its current production or update process, not by a pure origin story: a hand-authored prompt revised through measured search has entered the learned column for that update. The matrix distinguishes the axes conceptually; it does not assume that every quadrant has an equally scalable learning method.

The distinction rules out two symmetric positions. Weights-monism — the view that scalable learning happens only in distributed weights — goes beyond Sutton's production-method claim. That bounds what the 2019 essay establishes, not every later position Sutton holds.

A 2026 interview with Sutton and Khurram Javed acknowledges context as system state but treats continued weight updating as necessary for the structuring and generation of new concepts. The substantive dispute is therefore whether non-weight updates can supply the capabilities required for open-ended learning, not whether external state can change. Weight necessity is a serious empirical hypothesis, not a premise the lesson establishes by definition.

The systems cited here still deploy weights alongside harnesses, system prompts, and tools, so absorption at fixed task difficulty — absorption when the tasks assigned do not get harder as capability grows — has not yet made localized structure irrelevant. External structure can recur at a moving frontier when assigned difficulty rises with capability and some reliability function remains advantageous to externalize; the argument does not require such recurrence after demand saturates or the function is fully absorbed.

The second position, Hand-crafting-forever — defending localized forms by defending the manual production of their content — fails the lesson's own test. What scale removes is generalization whose scope was asserted rather than earned, and hand authorship is a common way to embed such scope. The compliant position is to test search and learning over localized forms too.

The machinery asymmetry explains the misreading

The cited cases suggest why the method axis is often conflated with parametric form: gradient descent supplies a computational evidence-to-update path together with credit assignment. Credit assignment is the problem of deciding which component bears responsibility for an outcome; the chain rule propagates that responsibility through parameter space. It need not expose a separately rejectable candidate, so proposal selection is one update architecture rather than the universal structure of learning. The localized quadrant has fragments with the artifact class fixed in advance — bounded instances of learned production over a single artifact class, such as prompt optimization, code evolution, and harness search — but no established method for a large, interdependent corpus. Before backpropagation scaled, hand-crafted features could likewise appear to mark a fixed boundary of learning because no general method reached them. That analogy motivates a search for the missing machinery; it does not show that such machinery must exist.

Why the form axis does not collapse into weights

Mixed deployments have reasons to retain external state even as parametric learning improves. Localized retention pays when sparse changes have bounded impact in a matching decomposition: explicit dependencies can bound the affected artifacts and checks, provided that the local advantage exceeds translation, routing, consistency, and coordination costs. Reproduction does not transfer authority, so a record's governance role survives content absorption, and a commitment exists nowhere until recorded. Enforced checks can also improve the selection environment for later candidates within their maintained domain, although overlap, drift, gaming, and maintenance costs can erase that gain.

These arguments establish persistent functions for localized state. They do not by themselves prove that learned semantic content must remain natural-language or symbolic rather than migrate into learned modular or parametric substrates. That stronger claim belongs to the scaling conjecture below.

The learned-localized quadrant and its missing machinery

The learned-localized quadrant already has bounded instances. FunSearch generates program functions with a pretrained language model, evaluates them, and retains successful programs for further search. AlphaDev used reinforcement learning and tree search to discover assembly sorting routines later incorporated into LLVM's standard C++ library. Their outputs are localized even though their search machinery includes parametric models.

Recent agent systems extend the pattern. One framework searches prompts, tools, and their composition as symbolic learnables; Memento-Skills continually rewrites structured Markdown skills as persistent evolving memory; and Co-Harness alternates harness search with fine-tuning, aiming to distill validated scaffolding into weights while keeping the harness revisable. Co-Harness is designed around cross-form coevolution, but its experiments do not establish that the allocation is efficient. Meta-Harness, an outer loop that searches task-specific LLM harness code, provides a precisely bounded result: its ablation compares a fixed summary-without-traces treatment with raw-trace access and leaves episode-backed theory untested.

These systems show that computation can optimize localized artifacts in bounded domains. They do not yet demonstrate efficient learning as corpus size, dependency density, and task horizon grow. The hard core is credit assignment without a chain rule: a deployment failure rarely identifies which artifact should change. Three discrete substitutes are visible in the methods cited here: explicit dependency edges bound the affected validation work, retained episodes carry attribution signals, and accumulated evaluation checks price candidate changes. No general way to compose them is identified here, while soft evaluation signals, supersession, bounded maintenance, and consolidation remain adjacent problems. Where a localized-artifact path uses proposal selection, it requires search, reject-capable evaluation, and operative retention. A direct update may collapse those roles, but it still owes an evidence-responsive operative change.

What may stay supplied

Compatibility with the lesson does not require every human-designed component to become self-modifiable. It distinguishes the human supply of task- or domain-specific competence from fixed machinery that implements a general production method over the reach being claimed.

Objectives, commitments, and grants of authority may remain supplied: evidence can bear on an objective without deriving the terminal objective itself, and nothing entails a commitment before it is made. Fixed general learning or search algorithms, metalanguages, runtimes, exact interfaces, resource controls, and trusted kernels may also remain. Their continued presence narrows what the claim covers, but fixed provenance alone does not make the method incompatible.

The pressure falls on family- or task-specific knowledge that people continue to construct as the system's claimed reach widens. A hand-built catalog of target solutions remains hand-supplied knowledge even when automatic retrieval selects among them. A general computational method that acquires the required production knowledge from permitted evidence is the stronger claim stated by the pressure for agentic factory development. Whether supposedly general machinery has hidden target-specific scope is empirical.

Adoption need not remain human. Computational closure asks whether any required decision on the declared path still needs a person, while warranted autonomy asks whether unattended evaluation is trustworthy. These are actor-allocation and warrant questions, not additions to the Bitter Lesson's production-method axis.

Compatibility is assessed per portion of a path

Because the columns classify parts and updates, compatibility with the lesson is a property of a portion of a path, not of a methodology or an artifact class. On a declared path, a portion has moved to scalable production when evidence-responsive computation determines and makes its update operative. In proposal-selection architectures, search proposes candidates and a reject-capable evaluator admits one; direct-update architectures may determine the successor without a separate gate. The retained output's form does not decide the classification.

The remainder is not thereby judged incompatible. Transfer prefers decisions it can warrant, so what stays human-supplied is often what no available evaluation can check, since warranted transfer leaves people the hardest-to-warrant decisions. Its difficulty does not reclassify the moved portion, because a method's ceiling bounds the method, not the transfer already made.

Two limits keep this from becoming a convergence claim. Evidence that several portions moved does not establish closure; closure separately requires every decision assigned to the declared pathway to be computational. The remainder can also be replenished while assigned difficulty keeps pace with capability and some function stays advantageous to externalize, since scaling absorbs scaffolding at fixed difficulty, not at the frontier.

A portion is worth having when it raises accepted outcomes per unit of total human effort, counting configuration, review, recovery, and repair — not when it reaches some share of the path, since a remainder small in count can be large in cost. In FunSearch, proposal and scoring moved while the specification, evaluator, skeleton, and function boundary stayed supplied; that bounds the demonstrated reach without erasing the moved production work.

The conjecture and the stake

The prediction is narrower than the conceptual matrix: for long-lived agent systems undergoing heterogeneous change, learning through more than one representational form can remain on the efficient frontier rather than serve only as temporary scaffolding. A serious test must compare learned localized methods with parametric learning and distillation baselines as corpus size, dependency density, task horizon, evaluation cost, and compute grow.

The bet can lose. If selection over localized knowledge remains artisanal as those dimensions scale, the strong learned-localized claim fails, even though interfaces, authoritative records, and checks may remain external. Commonplace, the agent-operated knowledge-base framework, is a human-assisted experiment in the missing loop: people still identify reusable lessons, assign blame, choose a form, and accept updates. Whether theory-mediated proposals improve sample efficiency remains an open bet. Reflective machinery must itself earn persistence rather than remain exempt by position.

Scope

  • "Fails", "the bet can lose", and "efficient frontier" throughout mean a worse frontier, not impossibility: a learned architecture with stable semantic modules, explicit scope, and localized update paths would confirm the mixed-form conjecture in a different substrate, not refute it.

Open Questions

  • Can the discrete credit-assignment substitutes — dependency edges, retained episodes, accumulated oracles — compose into a general method, or is per-domain assembly the ceiling?
  • Which supplied choices encode task-specific competence that scalable production must replace, and which are warranted general machinery or authority boundaries over the declared reach?

Relevant Notes: