JAZ exact-publication audit

Decision

hold — the public replay account needs the pinned implementation's sequential-invocation qualification. Concurrent invocations can select a sibling's saved response and latch divergence on the wrong invocation. This changes which retained code acquires execution authority, not merely replay fidelity. The synthesis presents invocation-local selection and withdrawal without this condition. A ledger reference to unspecified “parent-record limits” does not supply it.

This is one bounded independent audit under the next-61 commission and Audit a rerun. No integration or recovery was performed.

Identity and byte verification

  • Exact run: AAS-2026-10-15-jaz-a94bea79c61a-01.
  • Auditor: independent Pi worker; environment reports PI_PROVIDER=openai, PI_MODEL=gpt-6.1-sol, PI_REASONING_LEVEL=medium. No delegation.
  • Worktree: /home/zby/llm/commonplace/.commonplace/worktrees/jaz-a94bea79c61a.
  • Recorded command prefix: /home/zby/llm/commonplace/.commonplace/worktrees/jaz-a94bea79c61a/.venv/bin.
  • Method: a568c6dc489ad546374536b642c3889ce23149a8; verified worktree HEAD equality. Sibling preparation JSON is ready, token a94bea79c61a, same commit and prefix. Preparation records exclusion of dirty-origin changes.
  • Source: https://github.com/jaz-lang/JAZ, SRC-1, Git revision 0803d4971be785e95b80054b02259664d70fa3da.
  • Source root: /home/zby/llm/commonplace/.commonplace/worktrees/jaz-a94bea79c61a/related-systems/jaz-lang--JAZ. HEAD equals that revision; GIT_OPTIONAL_LOCKS=0 git status --short was empty.
  • Publication: worktree-relative kb/agentic-system-analyses/retained/jaz/. Its exact 13-file set and every SHA-256 match this run's JSON inventory row and effects/publish.json new-file map. The effect and state/attempts/000037-publish.json are completed.
  • Assembled ARTIFACT.yaml: 8377ad2b50098b77f737c18a13bf4a7ed4af8e4bf0286e4bc5ea093c4bb6af9a. Working type declaration: 093c6e4a6c7cab7e4e1a9db63f6d2d45eec6a24bf51f7b287dd1db929b388322. They are intentionally different artifacts.
  • Public overview SHA-256: 97d6ea1b07807dda259acd22600529e6b13c5596386e9837e942e237639563a5; synthesis: c1a6ec238d1ac27ecdd03bcb815932076b4383563d4e51156ed78f93f1588d4e.
  • Worktree status showed only untracked kb/agentic-system-analyses/retained/jaz/; no tracked method changes appeared. No origin incumbent is assigned in the inventory; this audit does not choose a replacement.

Method and historical boundary

I read the complete commission, audit procedure, earlier substantive commission, repository instructions, workshop collection contract and framing, and only the assigned exact-run inventory row. I compared the complete public overview and synthesis with kb/agentic-systems/reviews-archive/jaz.md, then followed the replay and workflow-export discrepancies into SRC-1 and selected actual worker messages. I also read boundary, reconciliation and report verification, plus relevant runtime, memory and epistemic records. This is not a second whole-system analysis or complete profile audit.

The historical review identifies run AAS-2026-09-26-jaz-01, a lowercase GitHub locator, and the same revision string. These metadata are comparison leads, not independent certification of historical source bytes or source equivalence. I did not use its linked old exact result as a second analytical authority. Current source reads decide the finding below.

The historical account's principal distinctions survive: execution versus correctness; host capabilities versus containment; raw-worker recursion limits; live console objects versus helper text; executable trace substitution versus reference retrieval; workflow generation versus demonstrated improvement; and external training versus installed learned weights. Its explicit sequential-stack warning prompted the bounded source check. The new synthesis's stronger withholding of reflection and improvement is an evidence qualification, not an established coverage regression requiring restoration of the old wording.

Consequential discrepancy

H1 — Replay's per-invocation guarantees omit a known concurrency counterroute

Source evidence. Under the frozen source root, src/jaz/hooks/builtin/replay.py:393–400 explicitly states SEQUENTIAL INVOKES ASSUMED: one _invoke_stack supplies the active invocation, whereas concurrent ainvoke execution can put another invocation on top. It names both sibling-queue mis-serving and latching another invocation's diverged flag. The inspected implementation independently supports the mechanism:

  • on_invoke_enter, lines 509–522, chooses the next child of the stack-top parent and pushes it.
  • on_invoke_exit, lines 524–526, pops only when the exiting ID is on top.
  • on_llm_query_send, lines 536–541, uses _invoke_stack[-1], without selecting the node by the query event's invocation ID.
  • The following send path checks that selected node's latch and supplies its next response. Optional edit comparison compares edit signatures, not an invocation-ID routing proof.

This is static implementation evidence, not an observed failing concurrent replay. No execution is needed to establish that the source's explicitly declared condition belongs beside the general mechanism claim.

Published mismatch. Synthesis, “What survives and acquires later authority,” says invocation-tree position/cursor select saved responses and optional divergence checks withdraw future substitution. Its limitations table addresses projection equivalence and replay/export fidelity, but not wrong-invocation routing under concurrency. runtime.md#rt-rte-replay-response similarly presents child selection and per-invocation live fallback without the sequential condition. epistemic.md#epi-rte-replay-check says withdrawal is per invoke and not automatic sibling withdrawal. Its authority-ledger row says “stack concurrency remain parent-record limits,” but the parent runtime record does not actually state that limit. The memory replay annotation also does not supply it.

The omitted condition matters because the saved response can direct executable Python. A reader could infer that interleaved invocations retain the described invocation-local queue and withdrawal semantics, even though the source says they do not. Generic “not exact-environment reproduction,” unobserved-operation and no-benefit disclaimers do not describe this specific implemented routing failure.

Unblock condition. Through a separately authorized publication correction/reanalysis route, carry the sequential-nesting condition and concurrent sibling-queue/divergence misattribution into the public synthesis and relevant replay records. Align the dangling ledger qualification with an actual qualified parent record. Alternatively, supply bounded frozen-source evidence establishing why this source-declared counterroute cannot occur within the reviewed scope and explicitly narrow the public scope accordingly. A live success demonstration alone would not remove the known condition. Revalidate, record the new byte map and obtain a fresh review of the changed publication; this report cannot clear altered bytes automatically.

Other selected challenges: qualifications already public

  • Seed prompt is not fully compared. Historical wording is more explicit, but source replay.py:339–343 and _edits_diverge at 619–631 support a limited edit-signature comparison rather than complete seed equality. The new memory annotation, epistemic replay record and synthesis already withhold full prompt/environment equivalence. No separate hold on this point.
  • Generated workflow is not an accepted execution checkpoint. workflow_replay.py:230–255 captures Enter-time code and writes completed exec-result records; lines 322–339 emit code under exception wrapping or as-is, rather than an acceptance certification. The new runtime and synthesis explicitly retain early-capture, refused-code, partial-artifact and governance-restoration limits. The historical review's coarser successful-workflow warning has not been lost. I did not execute generated code or fully retrace the validator/core cross-route.
  • Source classification revision is retained. A real semantic profile refusal was followed by a correction to partial original-input coverage, not silently treated as deterministic failure. The synthesis explicitly retains unresolved origins/content of merged inputs/scope. This is not another public qualification missing only in the audit.

Selected execution evidence

Trace: /media/zby/2b5e98b6-baf0-4e97-90d2-03321ea23372/outputs/batch-13/jaz.output (inventory size 361,822,043 bytes). I streamed the JSON and used final tool_execution_end worker message arrays, not repeated stream snapshots, as the selected evidence. Outer line numbers below are one-based JSONL locators. Each named inner call is paired with a role=toolResult of the same toolCallId in the same worker.

  1. Outer lines 2252/2384 launch/return report-verification attempt 000022, explicitly at this worktree with gpt-6.1-sol, medium thinking. The final worker array contains 88 tool calls and 88 matching results. Selected actual reads:
  2. call_1lOglvTWsaHKkP13E1F3djgX|fc_087660c06a0befe1016ad05178b3e487d2864b5c62a3abc8c4: read of SRC-1 replay.py, offset 509, limit 186; matching successful result contains stack-top selection and saved-response supply.
  3. call_VUw6Alo9uHkJAw26NQyJ553N|fc_087660c06a0befe1016ad05178b3ec87d2ac3a6fa5729646a2: read of replay.py, offset 773, limit 110; successful result contains active-stack-node divergence handling.
  4. call_SnyNhCZE1L3nPYzwaUr7v1sn|fc_087660c06a0befe1016ad05178b3cc87d2b56b2b0f57b5bdc5: read of workflow_replay.py, offset 142, limit 200; successful result contains capture and export branches.
  5. call_fAZm9HzW9PthfBiaGaVqVcJf|fc_087660c06a0befe1016ad051e9a1fc87d29f6ad6218ade0f64: write of verification output, with successful write result. Its “no material correction” conclusion therefore followed actual source inspection, but missed H1; source reads are not themselves semantic certification.
  6. call_39mXLq5YNEc66FPK6tW1GGGw|fc_087660c06a0befe1016ad0521e3d8487d2a8d86d5faf4948e1: actual commonplace-run validate-output call; result: Member content check passes; invocation residue not checked. This is not job acceptance or claim support.
  7. Outer lines 2934/3001, profile-verification attempt 000026: actual write/result pair call_yJrUfSUZJOgvYpSowZAU5wDF|fc_0281e32dbeb513d0016ad053ab584c87d2a4fcace0cc70d384 writes the objection. Retained judgment state/judgments/000027-apply-profile-verification-1.json refuses profile version 2ed507f1658476f7c34f1719f6d175912716e74e894ead14b64ec739690ca325: workflow original inputs/scope are not completely classified, so that unit cannot be known.
  8. Outer lines 3246/3313, correction attempt 000028: call_9tfmcgY4euRZfkVdoWOPpnp7|fc_00bb6dd6195bbf4a016ad0544243ec87d2a225fa0a7df770a3 actually edits the output profile; the matching result confirms four replacements, including known → partial. call_i6SDP0FiRg7Jy4Bze9t5TWcJ|fc_00bb6dd6195bbf4a016ad0544c444087d29c8fdbbe4303febe writes the correction answer successfully. The new profile hash is the published 39d4cb5f…; a later profile verifier accepts it. This is an observed correction, not a proposed edit.

A scan of retained judgment outcome fields found this one refused judgment; the other retained judgments are accepted. Repeated runtime/memory deterministic checks do not imply repeated semantic revisions. I did not exhaustively audit every worker, input read, write scope or hidden provider operation. No consequential scope violation was established in these selected pairs; absence elsewhere is not certified. Historical claims were not fed into any source-first worker.

Validation

From the prepared worktree, using its exact recorded prefix, I ran:

/home/zby/llm/commonplace/.commonplace/worktrees/jaz-a94bea79c61a/.venv/bin/commonplace-validate kb/agentic-system-analyses/retained/jaz

Exit 0; actual output:

VALIDATION SUCCESS
Target: kb/agentic-system-analyses/retained/jaz
Scope: kb/agentic-system-analyses/retained/jaz
Files analysed: 1
Text files: 0
Diagnostics: 0 failures across 0 subjects; 0 warnings across 0 subjects

Details: commonplace-validate --full kb/agentic-system-analyses/retained/jaz

The validator reports one assembled artifact, not 13 independently reported file validations. It emitted no notices, including no source-unavailable notices. This result is not quotation verification or a semantic resolution of H1.

From the same worktree I also ran the same prefixed commonplace-validate with absolute target /home/zby/llm/commonplace/kb/work/agentic-system-reruns/audits/next-publications/AAS-2026-10-15-jaz-a94bea79c61a-01.md. Exit 0: VALIDATION SUCCESS; target and scope both that absolute path; Files analysed: 1; Text files: 1; Diagnostics: 0 failures across 0 subjects; 0 warnings across 0 subjects. No notices were emitted. This is implicit workshop-text validation, not semantic acceptance.

Limits and parent action

No new source, acquisition, target execution, state-writing inspect/advance/recovery, publication/method/source edit, staging, commit, cleanup or integration was used. Read-only Git status used GIT_OPTIONAL_LOCKS=0. The only authored artifact is this report. No external source equivalence, observed replay, beneficial training or exhaustive worker compliance is inferred from publication success.

Stop at H1 rather than expanding to concurrent execution tests or whole-system reanalysis. The parent owns scheduling, authorized correction/recovery choice, full review, byte rechecks and integration. Integration remains blocked on H1's public qualification and subsequent review.