deploy-time-learning

Type: types/tag-readme.md

Deployment brings software into contact with users, changing needs, and operating conditions that design and testing did not fully anticipate. Assign this tag to work on what that use reveals or how people and systems respond: diagnosing failed requirements, modifying software or instructions, evaluating and retaining lessons, or preserving the knowledge needed for later change. Mechanisms and limits of learning from use qualify even when the argument generalizes beyond a particular deployment. Generic improvement or a passing mention of deployment is insufficient; adaptation to experience, new demands, or failures encountered in use must be substantive.

Responses may be human maintenance, automated adaptation, or a combination. Self-improving-systems covers systems that change their own organization; continual-learning covers continued learning through retained changes outside model weights. Work on those mechanisms as responses to deployed experience may carry both tags. This tag does not require autonomous change, weight updates, or a particular retention mechanism. A child of learning-theory.

What deployment reveals

• changing requirements conflate genuine change with disambiguation failure — the core case: some post-release change is world change, some is late discovery that the wrong interpretation of an underspecified requirement was built
• exact implementation does not validate a requirement against its objective — a requirement stays a conjectured proxy for the objective until use tests it
• use tests a decomposition locally; retained rationale makes transfer testable — running a design confirms only that it sufficed here
• system use is an initial selection environment when theory fit lacks a fixed oracle and system use provides evidence of theory fit, not independent warrant — the same phenomenon for a working theory: live use selects what fits, without licensing the claims
• Naur's compiler case tests one historically bounded documentation-and-consumption system — the classic account of why maintainers who lack the program's theory cannot make the changes deployment demands
• abstract an experience only when you can state where the lesson stops — what to retain from a deployment episode: a lesson only when its boundary is statable, else the instance

Where the change lands

• the deployed system, not the model alone, is the unit of learning — prompts, retrieval, tools, and runtime policy jointly determine behavior, so post-deployment change reaches all of them
• ad hoc prompts extend the system without schema changes — an LLM layer absorbs new requirements in natural language before the deterministic base changes
• constraining during deployment is continuous learning — prompts, schemas, tools, and tests accumulate the adaptation outside weights
• retained system-definition artifacts enable persistent deployment-time adaptation — the framework answer: retained, evaluated artifact changes give a deployed system a persistent adaptation path
• scaling absorbs scaffolding at fixed task difficulty, not at the deployment frontier — stronger models do not remove deployment-specific structure while assigned difficulty keeps pace with capability

Related Tags

• self-improving-systems — systems that make the post-deployment changes themselves: update architectures, reflection, and the actor allocation between humans and computation
• constraining — the mechanism by which a deployed system's interpretation space is narrowed as use reveals what it should have been