pm-llm-wiki synthesis

Type: agentic-system-analyses/types/agentic-system-synthesis.md

Bounded synthesis

A maintained product account, operated by an external agent

pm-llm-wiki is a prompt-defined product-management knowledge system. A human product manager (PM) curates sources, requests work and decides protected changes. An external coding agent interprets the instructions, synthesizes evidence and writes local Markdown. The repository supplies the maintainer schema and a complete fictional FitLoop pilot, not an executing agent host. This account covers the whole shipped knowledge loop at commit 64fb748ac9d187be4c51636aa56bfee1d9e5b116, with an applicability cutoff of 2026-11-09. SRC-1 supplies doctrine/design and reported operation only; the pilot's files are inspectable examples, not authenticated execution or real customer evidence (boundary, Source register).

Setup copies the schema into a product folder as CLAUDE.md or AGENTS.md, adds product context and creates sources, wiki, index and log. The external agent supplies model interpretation and file tools. Its provider, exact model version, permissions, scheduling and context-window behavior are unspecified. The ordinary runtime map is PM request → instruction and retained-file reads → agent synthesis or review → freely permitted writes or protected proposals → answer, maintained files and operation journal. Query/file-back, weekly lint and need-triggered schema progression are separate requested routes, not a shipped scheduler (RT-CMP-llm-agent, RT-OBJ-schema, runtime account).

From selected evidence to retained beliefs

The central separation is between source evidence and maintained product interpretation. Source notes remain read-only by policy. Problem pages accumulate evidence; assumptions retain beliefs, for/against evidence and history; decisions retain alternatives, rationale and reversal conditions. Questions preserve unresolved work, while an editable overview summarizes the current picture. These are reader-interpreted files, not a database with executable status rules (MEM-OBJ-sources, MEM-OBJ-problems, MEM-OBJ-assumptions, MEM-OBJ-decisions, MEM-OBJ-open-questions, MEM-OBJ-overview).

An ingest returns takeaways, checks existing beliefs and decision conditions, and updates relevant pages, index and log. A source that changes nothing takes a log-only branch. High-volume exports have a distinct admission step: the PM supplies a digest or approves one drafted by the model before ordinary synthesis. The pilot digest contains themes, counts and a quote, not the original exports; subsequent pages and the PRD consume that reduced evidence (RT-RTE-source-ingest, MEM-RTE-digest-admission, MEM-OBJ-digest).

Admission differs by operation. Ordinary problem, overview, question, catalog and journal updates do not require per-content PM approval. Belief-status changes, decisions, supersession and schema edits require a direct PM reply in the current conversation; relayed approval is insufficient. The explicit permission for unchanged-status assumption evidence conflicts with a broader final prohibition, and new-entry admission is underspecified. The pilot's conservative approval does not settle those rules. All these controls are policy-strength instructions interpreted by the agent, not authenticated authorization or filesystem enforcement. Arbitrary manual or host-tool writes are not mediated by repository code; that fact does not establish an exercised bypass (RT-RTE-source-ingest, RT-RTE-stage-upgrade).

There is a specific evidence-consuming criterion for problem framing: confirmation requires at least three independent sources or two source types; addressed status requires measured effect of a shipped change. This is not a causal-identification test. PM permission for a protected write is also not a general truth-acceptance criterion. Citation makes a statement inspectable but does not authenticate imported evidence or validate an inference (EPI-RTE-problem-confirmation, RT-CLM-evidence-control, epistemic bounded conclusion).

Reuse, delivery and review

A later query reads the index first, then pages judged relevant, with raw sources still reachable. Catalog summaries influence selection before full-page inspection. This is request-driven retrieval and a prescribed way to limit reading, not an embedding ranker, hard token budget, conversation compactor or isolated worker frame. Changed local instructions have a separate intended reload path: product pages supply evidence and rationale, whereas the schema supplies standing maintenance directions (MEM-OBJ-index, MEM-BAP-wiki-synthesis, RT-BAP-schema-agent, RT-RTE-query-fileback).

The representative July 2–3 pilot sequence connects these mechanisms. GymJoy launch intel is ingested, but the active decision's reversal condition requires an early and demonstrably strong launch. The example describes an early but shallow release, so the decision remains active. The second intel ingest triggers a competitor-entity proposal. While that and an evidence-addition proposal are pending, the agent reports drafting a PRD in-message. A coordinator's relayed approval leaves proposals pending; the journal later reports direct PM approval, schema/page updates and PRD file-back. This is a coherent reported workflow, not evidence of a spawned coordinator runtime or authenticated approval (workflow account, RT-RTE-stage-upgrade).

Delivery has two boundaries: the conversational draft and the subsequently cataloged exploration. Filing does not approve the product plan; the retained PRD remains a draft, and no later PRD-consuming invocation is supplied. Recovery in the illustrated approval case means preserving pending state and waiting for the PM. Lint similarly turns contradictions, gaps or triggered reversal conditions into questions or proposals, not autonomous decision reopening. Decisions are superseded rather than erased; stale pages are flagged rather than automatically expired. Crash recovery, locking and transactional rollback are not established (MEM-OBJ-prd, RT-RTE-query-fileback, RT-RTE-wiki-lint).

For a PM drafting under uncertain evidence, the example supports a useful distinction within the same answer: A2's correlation-only status reaches a proposed holdout rollout, but the PRD's categorical exclusion of price as a cause exceeds A1's weakened status and Q2's open hypothesis. The catalog also presents that price interpretation categorically. Likewise, a competitor's asking price does not establish willingness to pay. These are concrete limits on downstream warrant, not grounds for a system-wide epistemic grade. Source-linked retention and qualifier preservation are nonuniform; neither filing nor repeated summaries create independent corroboration (EPI-OBJ-prd, EPI-OBJ-index, EPI-OBJ-gymjoy, EPI-CLM-defensible-claims).

What changes through use, and what remains unproved

The strongest supported learning-related contribution is a designed path from product-interaction records, through a PM-admitted digest, into model-admitted durable wiki updates intended for later answers. This supports claimed trace-fed adaptation despite manual digest admission. It does not establish improved capacity for future action. The original exports are too coarsely described to classify their trace content; journal-to-method learning and other candidate paths remain unresolved (MEM-OBJ-digest, MEM-RTE-digest-admission, MEM-BAP-wiki-synthesis, profile verification).

The runtime's four theory-builder conditions have separate claimed, not observed, support in the ingest example: (1) named whole beliefs localize content; (2) instructions and the PRD describe decisions depending on it; (3) A1 receives identified counter-evidence and weakened reliance; (4) that result persists into the later illustrated round. This is product-belief criticism, not demonstrated criticism of the maintenance method. No comparison establishes that criticism improved future capacity (RT-RTE-source-ingest).

Reflection has separate design-level support: stage progression represents the system's entity structure, need pressure can change that representation, and revised instructions are intended to affect later maintenance. Exercised feedback is unobserved. Autonomous theory building is not attributable to this loop because PM decisions deliberately own protected internal admissions. Reflective theory building remains unestablished because methodological criticism against operation records is not shown. These limits do not negate localized product-belief revision (RT-RTE-stage-upgrade).

Schema progression also supplies a dormant intended authority path for self-change, but not exercised self-improvement at this boundary. Over the illustrated upgrade and intended subsequent sessions, no post-change operation establishes causal dependence on the extension, and objective-responsive improvement evidence is missing. A predeclared expansion trigger is not by itself evidence of criticism or success. Post-upgrade traces connecting revised instructions to later behavior, and comparisons against an independently specified improvement objective, would materially change this assessment. Authenticated approval/write traces would change enforcement and operation assessments; original exports and checked derivation chains would change fidelity assessments; actual product experiments would change causal conclusions. None is supplied here (RT-RTE-stage-upgrade, report verification).

Limitations

Limitation	Affected SRC IDs and record citations	Inspected boundary	Conclusion prevented	Evidence that would resolve it
Prompt policy and fictional reports are not observed operation.	SRC-1; RT-CMP-llm-agent, RT-RTE-source-ingest, RT-RTE-query-fileback	Entire shipped prompt-and-state loop; external host excluded.	Reliable gate adherence, authenticated approvals, actual downstream activation, real-product benefit, measured maintenance cost or provider-specific behavior.	Host implementation/contracts and authenticated sessions, downstream consumption and outcome evidence at an explicitly extended boundary.
Assumption admission rules are ambiguous.	SRC-1; RT-RTE-source-ingest, MEM-OBJ-assumptions; report verification limits	Template and pilot instructions: detailed unchanged-status exception, blanket prohibition and new-entry treatment.	A universal per-content approval rule or uniformly autonomous register admission. Status-change gating remains supported.	Reconciled instructions explicitly covering evidence-only and new-entry writes, with corresponding operation evidence.
Startup context and repeated intel are not synchronized.	SRC-1; RT-OBJ-schema, MEM-OBJ-gymjoy, MEM-OBJ-overview, MEM-OBJ-problems; report verification limits	Waitlist-era pilot startup context versus shipped-state pages; linking spec versus repeated details.	One current picture reaching every task, complete normalization or implemented deduplication. Relevant-page reads and the visible schema addition remain supported.	Updated context, explicit precedence/refresh rules and demonstrated normalization and subsequent reads.
Digest originals and count semantics are incomplete.	SRC-1; MEM-OBJ-digest, MEM-OBJ-problems; report verification limits	Weekly digest, adaptivity page and journal; fourteen digest mentions versus four page mentions.	Independently verified sample sizes, complete selection, digest fidelity, full lineage or original trace-source classification. The subsequent theme-reuse path remains supported.	Original exports, item-level mappings and explicit counter definitions.
Citations and filing do not settle inference warrant.	SRC-1; EPI-OBJ-prd, EPI-OBJ-index, EPI-OBJ-gymjoy, EPI-CLM-defensible-claims	Fictional PRD, catalog, competitor implications and broader README claims.	Universal defensibility, causal price/adaptivity conclusions or epistemic acceptance of every retained claim.	Claim-specific evidence-consuming review preserving scope and qualifiers; causal tests where causal conclusions are sought.
Persistence and intended feedback do not demonstrate improvement.	SRC-1; RT-RTE-source-ingest, RT-RTE-stage-upgrade, MEM-OBJ-log	Product-belief histories, PM-admitted schema extension and intended later reload.	Improved future capacity, reflective methodological criticism, exercised self-improvement or successful improvement.	Retained criticism linked to revisions and later consumers, post-change behavior causally dependent on the update, and objective-based comparisons.

Source-read log: none. This synthesis uses only the supplied accepted members; no source ambiguity required additional inspection.