deploy-time-learning

Type: kb/types/tag-readme.md

The organizing framework of the learning-theory area: deployed systems can retain evaluated changes to behavior-shaping artifacts — durable, inspectable, verifiable — giving them a persistent adaptation path across sessions that requires no weight update. Notes here cover the framework itself, the learning fundamentals it rests on, and the feedback signals that govern its quality. A child of learning-theory.

The framework

• Retained system-definition artifacts enable persistent deployment-time adaptation — the framework claim: retained, evaluated artifact updates give deployed systems cross-session adaptation without weight updates
• the verifiability gradient — the ladder deploy-time artifacts sit on, from restructured prompts through schemas and evals to deterministic code
• readable-artifact loop is the tractable unit for continual learning — the loop that makes behaviour change cheap: readable system-definition artifacts revised in place
• treat continual learning as representational-form coevolution — the system and its knowledge substrate evolve together rather than one training the other

Learning fundamentals

• learning is not only about generality — accumulation as the basic operation, with explanatory-reach as its key property; capacity decomposes into generality vs a reliability/speed/cost compound
• LLM learning phases fall between human learning modes — warns against literal human-LLM learning analogies
• in-context learning presupposes context engineering — "no continual learning needed" relocates the learning to the system layer rather than eliminating it
• choosing what to learn requires both validity and learning-value gates — accumulation policy: true is necessary but not sufficient
• abstract an experience only when you can state where the lesson stops — the abstract-vs-preserve decision: generalize only when the lesson's boundary is statable, else keep the instance
• use tests a decomposition locally; retained rationale makes transfer testable — the design-act case of the same boundary problem: the reasoning leaves no residue in the product, so it is retained at design time or it is gone

Feedback and signal quality

• changing requirements conflate genuine change with disambiguation failure — short iterations bound interpretation-error propagation, not just change-response latency
• evaluation automation is phase-gated by comprehension — comprehension and specification must precede optimization, or automation amplifies the wrong objective
• diagnostic richness constrains outer-loop learning quality — what the learning loop can learn is bounded by what its diagnostics distinguish
• final task success does not establish intended-path health — completion without path evidence cannot show whether the prescribed path stayed healthy

Related Tags

• constraining — the primary hardening mechanism inside the framework
• discovery — the operation that produces the framework's highest-explanatory-reach artifacts

Other tagged notes

• A compact, refreshable whole-picture narrative can replace infeasible fragment reconciliation - Holistic rewrite shifts reconciliation from each consumer to the author, but only when the whole-picture narrative can fit within effective context and be refreshed before the narrative goes stale
• A failure explanation becomes search control only when it changes a later branch decision - An explanation of a failed branch becomes operative search control only when its retention changes a later choice about scope, priority, probing, continuation, or abandonment
• A hand-crafted bootstrap fits the Bitter Lesson only if learning can outgrow it - A hand-crafted starting state fits the Bitter Lesson only if scalable learning displaces the task- and family-specific production knowledge it supplies as claimed reach widens
• An experiment identifies only the contrast it actually runs - Why missing comparisons, bundle-to-component attribution, and adjacent unrun treatments all overstate causal conclusions beyond an experiment's observed contrast
• An omitted improvement-loop function and a frozen one need different repairs - Five proposal-selection systems expose frozen functions, while a direct-update contrast shows why absence of a gate is not omission; HyperAgents supplies a preliminary partial unfreezing
• Continual learning requires governing behaviour-changing writes, not just storing content - For deployed systems, persistence is insufficient; continual learning must select, validate, authorize, and coordinate behaviour-changing updates across the representational forms a system can change
• Instantiation alone cannot model agent learning across sessions - The class/instance analogy captures session startup but omits the retained update relation that can revise later agent definitions and reusable-content placement
• Learning inside a fixed decomposition inherits its mistakes - Why optimization cannot repair consequential distinctions, responses, or mappings outside the effective update space of a fixed task decomposition
• Links encode conditional possibilities, not obligations - Links encode conditional possibilities, not obligations — every label must name a specific reader-need (the condition under which following pays off); content required for all reachable readers should be inlined, not linked
• Localized retention pays when sparse changes have bounded impact in a matching decomposition - Addressable retention localizes a sparse change when units match its decomposition; total adaptation stays local only when the affected units also have a small, explicit impact closure
• Machinery persists by warrant, not position, in a reflective loop - Reflection makes selected production machinery challengeable, but placement alone neither warrants nor requires revision; fixed general machinery may persist when its role and scope are earned
• Oracle accumulation improves selection for later candidates in its maintained domain - A failure retained as a lesson helps tasks that retrieve it; retained as a maintained check it improves selection for later candidates in its domain and amortizes validation
• Psychology-to-agent transfer needs per-principle failure-mode testing - Brainstorming a methodology for evaluating cognitive-science-to-agent transfer — assembled from three existing KB notes and tested against Youssef's five psychology principles as worked examples
• Reflective theory refinement has separate structural, epistemic, and implementation lineages - No single predecessor is closest to reflective theory refinement: runtime self-modeling supplies the self-target, classical theory refinement the mechanism with different fillers, and Workspace Optimization only an implementation analogy
• Scaling absorbs scaffolding at fixed task difficulty, not at the deployment frontier - Stronger models shrink the scaffolding a fixed task needs; durable deployment-specific structure recurs at the frontier only while assigned difficulty keeps pace with capability and some reliability function stays advantageous to externalize
• Soft degradation can bind before the hard cap even when required evidence fits - For quality-sensitive agent work whose required evidence fits within the provider window, volume, complexity, and interference can silently constrain usable context before the hard cap
• Soft-bound traditions as sources for context engineering strategies - Survey of twelve soft-bound traditions as candidate sources for context engineering strategies, with a three-tier assessment of what transfers, what's plausible, and what's blocked
• System use is an initial selection environment when theory fit lacks a fixed oracle - When no complete fixed oracle decides whether a claim belongs in a working theory, distributed consequences of live system use can provide an initial selection environment
• System use provides evidence of theory fit and causal usefulness, not independent warrant - Consequences of using a claim in a live system can test its integration and causal usefulness, but independent factual, formal, source, or scope evidence is still needed for its warrant
• The Bitter Lesson defense portfolio has one load-bearing member for the form-only rebuttal - The production-method versus representational-form distinction answers only a narrow weights-only inference; theory-guided bootstrapping is a provisional first strategy under incomplete global evaluation, not a defense of continuing hand production
• The bitter lesson selects against unearned reach, not against structure - The lesson selects against claims whose reach was asserted rather than earned by a refuting test, not against structure or origin — theory search in readable forms is its own method; earned reach protects the claim, not its carrier
• The bitter lesson selects production methods, not representational forms - The lesson's axis is production method — hand-crafted versus search-and-learning — not representational form. Learned localized forms are therefore a coherent scaling hypothesis, with cross-artifact credit assignment as the decisive open problem
• The deployed system, not the model alone, is the unit of learning - Because prompts, retrieval, tools, and runtime policy jointly determine deployed behavior, model-only learning leaves consequential system choices fixed