Charting the knowledge-access problem beyond RAG

Type: note · Status: seedling · Tags: foundations

Discussion about RAG, filesystems, databases, and graphs often treats storage or interface choice as the main question. But those are only part of the design space. The deeper problem is: how should a knowledge system be structured so a bounded agent can find, judge, combine, and trust the information needed for a task?

This is a brainstorming note, not a settled position. The goal is to chart the subproblems so comparisons stop collapsing unlike things into the same bucket.

What success should be measured against

The goal is not retrieval accuracy in isolation. A knowledge system for an agent should improve contextual competence: not just answering a question, but helping the agent classify, plan, explain, and act appropriately under bounded context.

That widens the evaluation target. An access strategy might be good at exact lookup but poor at orientation. Good at landing on a page but poor at surfacing contradictions. Good at retrieving chunks but poor at helping the agent compose them into an argument. Once the target is contextual competence rather than search accuracy, "best retrieval system" stops being a single-axis question.

The parts of the problem

Substrate — where knowledge lives and what operations are cheap. Files make hierarchical browsing and exact reads cheap. Databases make scored queries and derived indexes cheap. Graphs make typed traversal and neighborhood operations cheap. This is the execution-substrate question, not yet the navigation strategy. Files beat a database for agent-operated knowledge bases is one answer at one stage of maturity, but it does not settle how an agent decides what to load next.

Pointers — what the agent sees before it commits to reading. Titles, descriptions, table-of-contents entries, index blurbs, link phrases, path names, and graph edge labels are all pointers. Their job is not to answer the question directly, but to make the follow/skip decision cheap. This is the design space described by agents navigate by deciding what to read next.

Navigation modes — there are several distinct tasks usually lumped together as "retrieval":

  • long jump: find a promising landing point from the whole corpus
  • local traversal: move from one relevant item to nearby ones
  • exact lookup: find the place containing a string, syntax, or API name
  • orientation: get the shape of an area before descending
  • synthesis setup: assemble the subset worth reading together

Search, hierarchy browsing, link-following, grep, and indexes support different mixes of these tasks. Link-following and search impose different metadata requirements names one important split, but there are probably more than two modes.

These modes also consume context differently. A representation that works well for exact lookup may work poorly for synthesis setup because effective context is task-relative and complexity-relative, not a fixed model constant. The same token count can be easy for one task shape and unusable for another.

Synthesis — some questions are not "which page contains the answer?" but "which set of pages must be reconciled to produce the answer?" This is different from landing on the right document. Once a question requires comparison, contradiction resolution, or a whole-picture narrative, the problem becomes one of prompt assembly or pre-distillation, not just retrieval. Evolving understanding needs re-distillation, not composition is one case of this broader pattern.

Maintenance — any navigation aid can go stale. A table of contents, index, abstract, or graph edge is valuable only while it still predicts what the target contains. This is why stale indexes are worse than no indexes. The more a system depends on pointers and distillates, the more it needs maintenance loops that keep them trustworthy.

Why the RAG framing is too narrow

"RAG" usually names one move: turn a question into a query, retrieve chunks, stuff them into context, answer. That is one routing and loading strategy. It is not the full problem.

The framing becomes misleading when it hides these differences:

  • selecting one landing point vs assembling a reading plan
  • retrieving chunks vs exposing structure the agent can traverse
  • finding candidate pages vs preparing a synthesizable packet
  • storing knowledge vs activating it in the right context

A filesystem interface can outperform naive RAG not because files are magical, but because it converts one hard global decision into many cheaper local ones: ls to orient, grep to narrow, cat to inspect, then continue. A graph can outperform both when typed neighborhoods matter. A database can outperform both when scoring and filtering dominate. The right comparison is task-relative, not ideology-relative.

The bounded-context reason for this is easy to miss. Since agent context is constrained by soft degradation, not hard token limits, access strategies should be compared by how they manage degradation for a task shape, not by whether they match a preferred storage metaphor.

Distillation as navigation infrastructure

A table of contents is distillation for the task of search. So are abstracts, descriptions, index entries, overviews, and maybe graph neighborhood summaries. They compress a larger body so the agent can decide whether to descend.

This suggests that good knowledge systems may need multiple distillates of the same source, each aimed at a different navigation or synthesis task:

  • title: fastest coarse relevance check
  • description: search-result discrimination
  • TOC or index entry: orientation within an area
  • abstract or overview: mid-resolution relevance judgment
  • full text: detailed reasoning material
  • synthesized narrative: whole-picture consumption when composition would exceed effective context

If this is right, then the question is not whether semantic search is enough. It is whether the system provides the right pre-compressed views for the decisions the agent must make at each step.

Discovery and synthesis want different structures

Short composable notes maximize combinatorial discovery argues that the library should be optimized for co-loading many small independent claims. That is one access goal: discovery by juxtaposition.

But the same structure can be bad for consumers who need the whole current picture. When the task is onboarding into an evolving situation, resolving tensions across many notes, or carrying forward a current strategy, fragment composition can exceed effective context. In those cases, a synthesized narrative or workshop artifact may be the right access surface. This suggests that "knowledge access" includes a tension between discovery-optimized structures and synthesis-optimized structures, not just a choice of retriever.

Historical analogy

Libraries solved analogous problems long before computers. They did not rely on one universal access mechanism. They used catalogues, subject headings, shelf order, abstracts, bibliographies, and reference desks. The pattern seems durable: one corpus, multiple projections, each optimized for a different access task.

The broader pattern may be older and wider than libraries. Soft-bound traditions as sources for context engineering strategies identifies recurring responses wherever a bounded processor cannot consume everything at once: selection pressure, compression, progressive formalization, and modularity. Library systems, technical writing, hypertext, and faceted classification all look like members of this family.

What seems new in agent systems is not the existence of these layers but the hard bounded-context constraint. For an LLM, bad organization is not just inconvenient. It directly competes with the task for limited reasoning budget. That makes context engineering the umbrella problem and makes distillation part of the access architecture, not just a summarization afterthought.

Open questions

  • How many distinct navigation modes are there beyond link-following and search?
  • When does a question cross from retrieval into synthesis, and can that boundary be detected automatically?
  • What is the right division of labor between discovery-optimized library structures and synthesis-optimized workshop artifacts?
  • What kinds of distillates are worth maintaining by hand, and which should be generated?
  • When does graph structure earn its maintenance cost over indexes plus links?
  • Can a system measure whether its pointers actually improve read/skip decisions, rather than only final answer accuracy?

Relevant Notes: