Dynamic Cheatsheet boundary

Type: agentic-system-analyses/types/agentic-system-boundary.md

Boundary and evidence

This run analyses how Dynamic Cheatsheet creates, carries forward, retrieves and consumes inference-time memory. The target is the shipped repository system, not just the LanguageModel API. It is an in-scope memory/knowledge/context-engineering system: model-generated answers supply material for model-based curation, and retained text or previous input-output pairs become context for later model calls. The README declares this intended use (SRC-1, README.md):

A lightweight framework that gives language models (LMs) a persistent, evolving memory during inference time. --- README.md

The whole-system boundary includes the benchmark driver and interactive example as well as the library, its shipped prompts, provider client and evaluation consumers. The driver owns cross-example state, disk persistence and continuation; these responsibilities are not assigned to an excluded host. Initial inspection traced all six supported approaches in advanced_generate: default generation, cumulative curation, retrieval with synthesis, cumulative plus retrieval, full-history appending and direct retrieval. It also traced the generator's local-code continuation and provider-native code-execution branches (SRC-1, dynamic_cheatsheet/language_model.py, run_benchmark.py, text_generation/simple_unified_client.py).

Cumulative curation passes the question, generated response and previous cheatsheet into a curator call, then admits the extractor's returned text as the next cheatsheet (SRC-1, dynamic_cheatsheet/language_model.py, dynamic_cheatsheet/utils/extractor.py). This is an executable memory-update path, not solely a claim in documentation:

Extract the new cheatsheet from the output (if present); otherwise, return the old cheatsheet

new_cheatsheet = extract_cheatsheet(response=cheatsheet_output, old_cheatsheet=current_cheatsheet) cheatsheet = new_cheatsheet --- dynamic_cheatsheet/language_model.py

The shipped curator prompts are part of the system's maintenance contract. They request retention, refinement, usage counts and explicit copying of useful previous content. Their instructions to verify correctness are model-facing intent, not evidence of an independently enforced correctness gate (SRC-1, prompts/curator_prompt_for_dc_cumulative.txt, prompts/curator_prompt_for_dc_retrieval_synthesis.txt). The generator prompt supplies the consumption contract and answer/code-execution formatting (SRC-1, prompts/generator_prompt.txt).

The benchmark driver initializes memory, optionally reloads the last cheatsheet and previous outputs from JSONL, passes earlier state to the library, carries the returned cheatsheet into the next example and rewrites the output checkpoint after each example. The standalone example notebook also feeds returned cheatsheets into subsequent calls (SRC-1, run_benchmark.py, ExampleUsage.ipynb). The continuation path is explicit:

Load the previous cheatsheet from the last output (if available)

last_cheatsheet = outputs[-1].get("final_cheatsheet") if last_cheatsheet is not None: cheatsheet = last_cheatsheet

generator_outputs_so_far = [output["final_output"] for output in outputs] --- run_benchmark.py

The evaluation boundary includes online scoring in the benchmark driver and later scoring of retained JSONL in the evaluation notebook. This prevents treating the repository as only a stateless API or omitting its shipped persistence/reload/evaluation loop (SRC-1, run_benchmark.py, EvaluatingResults.ipynb, dynamic_cheatsheet/utils/evaluation.py).

Tracked top-level coverage

All rows refer to SRC-1 at the reviewed commit. “Not yet inspected” describes available evidence, not an access gap. Artifact contents remain available to later jobs at the same revision.

Frozen evidence and dependencies

The sole frozen source is SRC-1, repository https://github.com/suzgunmirac/dynamic-cheatsheet at full commit 5cfe3c37e8e52b1d858d0f3df46e7f17c50991b9. The acquired checkout's HEAD matches that commit and its status was clean at inspection. The applicability cutoff is 2026-10-12; later repository revisions and live services are outside this evidence. Overall evidence is code-grounded. No target execution, provider call or benchmark reproduction was performed.

External dependencies include the selected provider's SDK/service, OpenAI-compatible endpoints, Python and the local subprocess environment, NumPy/scikit-learn for retrieval, tiktoken, pandas, datasets, dotenv and typed argument parsing. The driver loads Game of 24 through the external turingmachine/meta-prompting dataset rather than a shipped local task bundle. These interfaces are visible in SRC-1, README.md, run_benchmark.py, dynamic_cheatsheet/language_model.py, dynamic_cheatsheet/utils/execute_code.py and text_generation/simple_unified_client.py; their external implementations and moving service/data versions are not frozen here. This prevents conclusions about present provider compatibility, remote execution isolation, exact external dataset bytes or complete environment reproducibility. Local execution source remains included; remote sandbox claims do not establish local sandboxing.

The linked paper is not captured or registered and is deliberately excluded. Paper-only experimental conditions and causal explanations cannot support this run. Repository result files remain eligible frozen evidence, but their presence or schema alone does not establish an observed memory benefit. No required shipped state-producing, updating, admitting, persisting, reloading or consuming source path was unavailable; the uninspected artifact payloads and auxiliary facilities above limit claims rather than turning the core loop into a partial boundary.

Source register