Case packet
Neutral case identifier: case-5ea996e957b487
The possible directed relationship from Artifact A to Artifact B is under review.
Artifact A
LLM context is a homoiconic medium
In the LLM context window, instructions and data share the same representation — natural language tokens. A system prompt, a user message, a tool output, and a piece of content being analyzed are all just text. There is no type-level distinction between "program" and "data." This is homoiconicity: the medium used to express programs is the same medium used to express the data they operate on.
Precedents
Lisp's homoiconicity. Code and data share the same representation — lists. A list is both a data structure you can inspect and a program you can evaluate. In an LLM-based system, a markdown file is both content you can read/link/analyze and instructions you can hand to a sub-agent for execution. Lisp macros — code that writes code — map to instructions that produce reports containing further instructions. Quote/eval — toggling between treating something as data vs executing it — maps to reading an instructions note vs handing it off.
Emacs as ad hoc extension culture. Emacs is written in Elisp — the system and its extension language are the same thing, so there is no boundary between built-in functionality and user extensions. The init file mixes configuration and ad hoc programs. The constraining trajectory is the same: inline snippet in init.el → extracted function → published package. The KB's equivalent: [ad hoc instructions note → extracted skill → registered tool].
Smalltalk's live image. The image is both the program and the development environment, and you modify the running system from inside itself using the same language. The KB has the same property — its methodology is written in the same markdown it operates on.
Other homoiconic languages. Prolog (programs are clauses in the same database as facts), Tcl (everything is a string, including code), Rebol/Red (code is data blocks), XSLT (XML transforming XML). All share the property that the boundary between using and extending the system is fluid.
What homoiconicity enables
The common thread across these systems: blurring the boundary between using and extending makes the system more adaptable than systems with rigid extension points. In the KB, [ad hoc prompts extend the system without schema changes] precisely because instructions and content share the same medium. No registration, no type system gatekeeping, no compilation step — write a markdown file and it's both content and executable spec.
What homoiconicity costs
The same lack of boundary that enables extensibility creates hazards:
Scoping failures have no guardrails. [LLM context is composed without scoping] — and homoiconicity makes it worse, because there is no structural way to distinguish "this is a binding the sub-task should see" from "this is leftover from an earlier computation." Both are just tokens. The [constraining gradient from instructions to scripts] is one response — codifying practices into deterministic code imposes structural boundaries that the homoiconic medium itself does not provide.
Prompt injection. The most direct consequence: if instructions and data are the same medium, data can contain instructions. This is the LLM equivalent of SQL injection, and it exists for the same structural reason — the program/data boundary is conventional, not enforced.
Discoverability. Emacs's .emacs files are notoriously personal and opaque, Lisp macros can make code unreadable, and ad hoc instructions notes are invisible unless you know they exist. When everything can be both program and data, it's hard to know what a system actually does.
Relevant Notes:
- [context efficiency is the central design concern in agent systems] — intensified by: homoiconicity means instructions and data compete as equal tokens, removing any priority mechanism for protecting instructions from dilution
Artifact B
Methodology enforcement is constraining
The ways we enforce methodology in the KB — instructions, skills, hooks, scripts — map directly onto the [constraining spectrum]. The enforcement layers parallel the [verifiability gradient] — where codification moves code from prompt tweaks through schemas to deterministic modules, methodology enforcement moves practices from written guidance through structured skills to automated scripts. Each layer trades flexibility for reliability by reducing two things: semantic underspecification (committing to one interpretation of what the practice means) and execution indeterminism (ensuring the practice fires consistently across runs). Moving from instructions to scripts progressively eliminates both.
| Layer | Trigger | Response | Reliability | Example |
|---|---|---|---|---|
| Ad hoc prompt | indeterministic (caller writes one) | underspecified + indeterministic (LLM interprets) | lowest | "read these three docs through this lens" in a one-off instructions note |
| Instruction | indeterministic (LLM remembers) | underspecified + indeterministic (LLM interprets) | low | "check descriptions" in CLAUDE.md |
| Skill | deterministic (user invokes) | underspecified + indeterministic (LLM executes) | medium | /validate checks note quality |
| Hook (warn) | deterministic (event fires) | underspecified + indeterministic (LLM acts on output) | medium-high | validate-note.sh outputs WARN on missing description |
| Hook (block) | deterministic (event fires) | deterministic (rejected) | high | exit 1 prevents the operation |
| Script | deterministic (user/hook runs) | deterministic (code runs) | highest | generate_notes_index.py rebuilds note listings |
[Ad hoc prompts] are looser than persistent instructions — they're one-shot, not loaded into every session, and exist only for a single use. They sit below instructions on the gradient because they add a third source of unreliability: the prompt itself is ephemeral, so it can't even accumulate the weak consistency that comes from an instruction being present every time.
Instructions have the lowest persistent reliability because both phenomena compound: the LLM may not remember to apply the practice (indeterminism in triggering), and when it does, it interprets the instruction through underspecified semantics ("check descriptions" admits multiple valid readings of what counts as a good description). Skills eliminate the trigger problem — the user invokes them deterministically — but the response is still an LLM interpreting an underspecified spec. Blocking hooks and scripts eliminate both phenomena entirely.
The key insight: hooks are not cleanly "deterministic." A hook that outputs a warning is a deterministic trigger with an underspecified, indeterministic response — the LLM decides what to do with the warning. Only blocking hooks (exit non-zero) are fully deterministic. This means the three-tier model (instruction → skill → hook) that arscontexta uses oversimplifies — the real picture is a gradient, which is just constraining.
Maturation trajectory
This is [progressive compilation applied to methodology] — new best practices should start as underspecified natural-language guidance and constrain toward precise, deterministic enforcement as they prove out:
- Instruction — write it in CLAUDE.md or WRITING.md. Cheap to revise, tests whether the practice is worth encoding. If the LLM follows it inconsistently, that's signal.
- Skill — encode it as a structured prompt. Reliable when invoked, but requires explicit invocation. Good for judgment-requiring operations that shouldn't be automated.
- Hook/script — automate the deterministic parts. Only after the practice has constrained enough that you know exactly what the check should do.
When to move down. The strongest signal for automation is when the agent consistently proposes the same correct next step — meaning both that the LLM has converged on a single interpretation of the underspecified spec, and that it executes it reliably across runs. If the LLM's response is predictable and always right, the prompt-to-action path is just overhead; a hook or script would do the same thing without the latency or token cost. This is the codification trigger: a pattern has emerged from repeated execution, and constraining it commits to that interpretation in precise code — resolving the semantic underspecification by design rather than by luck, and eliminating the indeterminism entirely.
Not everything should complete the trajectory. Operations requiring semantic judgment (like "is this connection genuine?") belong permanently at the skill level — their [oracle strength] is too low to support deterministic verification. Attempting to automate judgment produces confident systematic errors — the over-automation risk. The former Topics-footer sync script is a clean example of the trajectory completing: an LLM-generated footer was recognised as fully mechanical, and the operation moved to a deterministic script.
The trajectory requires active observation. The [context engineering study] found that 50% of AGENTS.md files were never changed after creation — write-once artifacts that never enter the maturation trajectory at all. The codification trigger above (observing that the agent consistently proposes the same correct step) only fires if someone is watching. Among the files that do evolve, additions (78 commits) and modifications (59) vastly outnumber removals (23) and section deletions (2) — pruning is a discipline, not an emergent behavior. Instructions accumulate unless someone actively removes them.
The maturation trajectory parallels [document type maturation] — just as documents start as untyped note and gain type information as they codify, practices start as written guidance and gain enforcement structure as they prove out. Both are gradual typing applied to different substrates: types accumulate verifiable structural properties; enforcement accumulates deterministic triggers and responses. The [loading frequency hierarchy] mirrors the same gradient from the information-delivery side — CLAUDE.md instructions, skill descriptions, skill bodies — but for loading specificity rather than enforcement reliability.
Current state
We have hooks in .claude/hooks/ but they aren't wired up ("hooks": {} in settings.json) and reference old paths. We have scripts that work (generate_notes_index.py). We have skills that work (validate, connect, ingest). We have instructions that work (CLAUDE.md, WRITING.md). The gradient exists — we just haven't needed to push anything further toward the deterministic end yet.
Open questions
- When should a WRITING.md instruction become a validate check? [Oracle strength] may provide the answer: a practice is ready to move down the gradient when you can cheaply verify whether it was followed correctly. If verification requires semantic judgment, the practice stays at skill level; if it can be reduced to structural checks, it is a candidate for scripting.
- Should hook warnings be treated differently from skill output? The LLM sees both as text, but the trigger mechanism differs.
- Are there practices currently at skill level that should be scripts? (
sync_topic_links.pywas probably this historically — a skill-level operation that turned out to be fully deterministic before the underlying Topics system was removed.)
Relevant Notes:
- [oracle strength spectrum] — determines when a practice is ready to move down the enforcement gradient: cheap verification enables scripting; expensive verification keeps the practice at skill level
Under-review context phrase
the constraining gradient from instruction to script imposes structural boundaries that the homoiconic medium itself cannot provide