A good agentic KB maximizes contextual competence through discoverable, composable, trustworthy knowledge

Type: note · Status: seedling

What is a good agentic knowledge base?

A knowledge base for an agent is not a search engine. An agent doesn't just retrieve facts — it classifies, plans, communicates, and reasons on behalf of a user. The accumulated knowledge must make all of these actions more competent. Claw learning is broader than retrieval names this directly: a Claw's learning loop must improve action capacity, not just answer-finding.

The criterion is contextual competence: the agent's ability to act appropriately given what it knows about the domain, the user, and the project. A good agentic KB maximizes contextual competence under the constraint of bounded context — the finite window through which the agent receives everything it can attend to.

This rules out "just put everything in the context." The constraint is real and structural: context degrades before it runs out (attention rot), complexity costs more than volume, and there is no scoping mechanism to isolate what matters from what doesn't. A KB that produces competence must produce it efficiently.

Three properties of knowledge that serves competence

For knowledge to improve an agent's actions under bounded context, it needs three properties. Each addresses a different failure mode:

Discoverable — the agent can find what it needs without loading everything. Failure mode: knowledge exists but the agent doesn't encounter it. Commonplace mechanisms: search over files as the universal interface (grep for pattern matching, frontmatter queries for structured fields, qmd for semantic full-text search — the base layer that makes everything else work), claim titles that carry the argument without requiring descent into the body, retrieval-oriented descriptions that differentiate notes from each other, area indexes that give the landscape at a glance, progressive disclosure that matches specificity to loading frequency.

Composable — pieces of knowledge combine into larger arguments and can be used as premises in new reasoning. Failure mode: the agent retrieves a note but can't use it with other notes — the knowledge is an island. Commonplace mechanisms: typed links with articulated relationships that encode how notes relate ("since X" is a premise, "but Y" is a tension, "extends Z" is a generalization), resolution-switching between indexes (broad) and note bodies (narrow), the composability check in WRITING.md ("can this note be linked without dragging irrelevant context?").

Trustworthy — the agent can rely on what the knowledge says without independently verifying it each time. Failure mode: the agent loads a note but can't trust its claims — the knowledge is noise. Commonplace mechanisms: the stabilisation gradient from raw text to structured claims to deterministic code, the type system that marks what kind of artifact something is, status fields that signal maturity (seedling → current), validation scripts that enforce structural contracts.

These three properties have a dependency structure. Composability depends on the other two: you can't compose what you can't find (discoverability), and you can't build on an unreliable premise (trustworthiness). Trustworthiness depends on discoverability: a note that can't be found can't be challenged, corrected, or kept current. Discoverability is the foundation — without it, the other two are inert.

Accumulation and three operations that improve these properties

The KB doesn't stay good automatically. It improves through learning, starting with the most basic operation.

Accumulation — adding knowledge to the store — is the most basic learning operation. But what you accumulate varies in reach. Storing a fact ("the key is on the table") is low-reach accumulation — useful for the immediate context but it doesn't transfer. Storing a theory ("systems that optimize for efficiency under normal load sacrifice resilience to overload") is high-reach accumulation — it applies in contexts it wasn't designed for. Both are genuine learning. Reach is the property that distinguishes them.

Three operations transform accumulated knowledge, each targeting different properties and operating on different dimensions of capacity:

Stabilisation constrains the interpretation space — from storing an output, to writing conventions, to extracting deterministic code. It trades generality for the reliability+speed+cost compound. Stabilisation primarily improves trustworthiness: a validated script is more trustworthy than a prose instruction, a structured claim more trustworthy than a raw observation. It secondarily improves discoverability — a note with good frontmatter is more findable than raw text.

Distillation extracts focused artifacts from larger bodies of reasoning — methodology notes become skills, workshop explorations become design principles, research becomes prescriptive notes. It also trades generality for compound gains, but through selection and compression rather than constraint. Distillation primarily improves discoverability: a distilled skill fits in one context load where fifteen methodology notes wouldn't. It secondarily improves composability — a focused note composes better than a sprawling exploration. Deletion and pruning are distillation's negative face — selecting what to keep implies selecting what to discard, and removing outdated or contradictory knowledge improves capacity by subtraction.

Discovery posits a new general concept and simultaneously recognizes existing particulars as instances of it. Unlike stabilisation and distillation, discovery creates generality rather than sacrificing it. It primarily improves composability: a note that names a shared mechanism ("systems degrade under structural overload") becomes a hub that connects previously isolated notes, making them available as premises in each other's arguments. It secondarily improves discoverability — once the mechanism has a name, recognizing further instances becomes cheap. The naming amortizes the discovery cost. Discovery also produces the highest-reach items for accumulation — theories are what accumulation stores when it's at its most valuable.

Operation Capacity trade-off Primary property improved Commonplace example
Accumulation Adds knowledge (reach varies) All three (depends on what's accumulated) Storing a new observation, recording a discovered principle
Stabilisation Generality → compound Trustworthiness text → note → structured claim → validation script
Distillation Generality → compound Discoverability 15 methodology notes → 1 /connect skill
Discovery Creates generality Composability Seeing that PL scoping and agent context loading share the same mechanism

A KB that only accumulates grows but never improves what it has — a pile, not a system. One that only stabilises gets rigid — trustworthy but unable to grow. One that only distils gets focused but shallow — discoverable but with nothing deep to discover. One that only discovers gets conceptually rich but unreliable and hard to navigate — composable in theory but not in practice. All four must operate continuously.

Reach: why some knowledge serves competence better than others

The three properties tell you what knowledge must be — discoverable, composable, trustworthy. They don't tell you which knowledge is worth making discoverable, composable, and trustworthy. A KB could satisfy all three properties while being full of shallow facts that don't help the agent reason in new situations.

David Deutsch distinguishes adaptive knowledge (patterns that work but don't explain why) from explanatory knowledge (accounts that capture causal structure). The distinguishing property is reach: explanatory knowledge applies in contexts it was never designed for, because the explanation captures structure that isn't context-dependent. A gene "knows" how to build an eye but can't help you design a telescope. Newton's optics does both.

Reach matters for an agentic KB because an agent encounters novel situations constantly — new tasks, new codebases, new user requests. Knowledge with reach transfers to these situations. Knowledge without reach doesn't. A note that says "files beat a database because agents already have Read/Write/Grep tools" is adaptive — it works now, but if the toolset changes, the conclusion changes unpredictably. A note that says "files beat a database because the universal interface principle minimizes coupling between the storage layer and the consumer" has reach — it explains why, predicts when the conclusion would change, and applies to storage decisions beyond this specific system.

Reach connects to accumulation and the three transformation operations asymmetrically:

  • Accumulation is where reach varies most. Reach is the key property of what you accumulate — facts sit at the low end (adaptive, context-bound), theories at the high end (explanatory, transferable). The same operation (adding knowledge to the store) produces vastly different value depending on the reach of what's added.
  • Stabilisation preserves reach but doesn't create it. A crystallised script is no more explanatory than the prompt it replaced — just more reliable. Stabilisation improves the compound (reliability, speed, cost) without affecting generality, and reach lives in the generality dimension.
  • Distillation can preserve or destroy reach. Distilling methodology into a skill preserves reach if the skill captures the principle, not just the procedure. A skill that says "do steps 1-5" has lost the reach of the reasoning that produced those steps. A skill that says "do steps 1-5 because X" keeps it.
  • Discovery is where reach is created. The discovery operation — positing a general concept, recognizing particulars as instances — is precisely the act of finding structure that reaches beyond the original cases. The three depths of discovery (shared feature → shared structure → generative model) correspond to increasing reach. Discovery produces theories — the highest-reach items accumulation can store.

The KB's first-principles filter is, in Deutsch's terms, a filter that selects for reach. When we require notes to derive patterns from constraints rather than recording "X works," we're selecting for explanatory knowledge that will transfer to new contexts. The programming fast-pass is a bet on reach — PL concepts apply to KB design because they capture structure that isn't programming-specific. Convergence evidence (Thalo independently building a compiler for knowledge management) suggests the reach is real.

Tension: reach vs action value

There is a tension between reach as the primary quality criterion and contextual competence as the goal. Claw learning is broader than retrieval argues that an action-oriented system needs preferences, procedures, and judgment precedents — knowledge types that are inherently low-reach (context-specific) yet high-value for action. A procedure like "always run validation before committing" has minimal reach but substantial competence value.

The tension is real but not a contradiction, for two reasons. First, under bounded context, a compact theory replaces many facts — one note explaining why a pattern holds is more context-efficient than twenty notes recording instances where it worked. Reach is practical, not just intellectually satisfying, because it compresses knowledge. Second, reach matters most when knowledge leaves the KB — when insights transfer to new projects, new agents, or new domains. A KB that only stores context-specific procedures is useful for its host system but produces nothing portable. One that also stores explanatory knowledge generates value beyond its immediate context.

The resolution is not to choose one over the other but to recognize that the four operations serve different needs: accumulation stores both facts and theories, stabilisation makes procedures reliable, and discovery creates the theories that give a KB reach beyond its origin.

Why this favors authored knowledge

The reviews of sift-kg and Siftly sharpen this theory by contrast. Both systems build knowledge structures from documents — sift-kg extracts entity-relation graphs via LLM, Siftly enriches and classifies bookmarks. Both are effective at what they do. But the kind of knowledge they produce — entity-relation triples, category assignments, semantic tags — is adaptive, not explanatory.

An extracted triple ("Person X works for Organization Y, confidence 0.85") captures a fact, not an explanation. It has no reach: it says nothing about why the relationship exists, doesn't predict when it would change, and doesn't compose with other triples to produce novel reasoning. The graph's value is in aggregation and navigation, not in the explanatory power of individual edges.

An authored note ("Context efficiency is the central design concern because context is the only channel and it degrades before it runs out") captures causal structure. It predicts consequences of change and composes with other notes to support novel arguments. Its value is in reach — it applies to design decisions not yet imagined.

This isn't a claim that extraction can never produce knowledge with reach — an LLM that extracted causal explanations from papers might. It's that the act of authoring, specifically the judgment that selects what matters and explains why, is what produces reach. Current extraction systems optimize for coverage and accuracy of facts, not for explanatory depth. Commonplace optimizes for explanatory depth — and that is the theory of why its arrangements work.

The theory, compressed

A good agentic knowledge base maximizes contextual competence — the agent's ability to act appropriately given accumulated knowledge about the domain, user, and project — under bounded context. It does this through knowledge that is discoverable (findable without loading everything), composable (usable as premises in novel reasoning), and trustworthy (reliable enough to act on without re-verification). Accumulation — adding knowledge to the store — is the most basic learning operation, and reach is its key property: facts (low reach) are adaptive, theories (high reach) are explanatory and transfer to new contexts. Three operations transform accumulated knowledge: stabilisation (constraining — improves trustworthiness), distillation (extracting and pruning — improves discoverability), and discovery (generalizing — improves composability, produces the highest-reach items to accumulate). The KB's first-principles filter selects for reach — not because procedures and facts lack value, but because theories compress knowledge under bounded context and transfer when knowledge leaves the KB. Authoring — the act of judgment that explains why — is the primary source of knowledge with reach.

Relevant Notes:

Topics: