Hidden-failures review: Claude Code sessions, main checkout, 2026-11-05 to 2026-11-07

Summary and decision

No delivered result has a demonstrated defect. Three hidden problems matter.

  1. A conclusion presented as exhaustive was based on capped output (I2, 25868e95). The operator was told "nothing recorded says the semantics work gates the machine. I checked every open workshop README". The prerequisite grep behind that claim ended in head -40 and returned exactly 40 lines, so it was probably truncated. The machine-side READMEs were read only to line 45. The reply did not mention either limit. This conclusion fed a prioritisation decision.
  2. An unguarded edit failed silently when another session wrote to the same file (I1, 3929c733 L173). A Python str.replace on scripts/scan_tools/README.md did nothing, because a concurrent Codex session had already rewritten the target text. The edit still exited cleanly. The agent noticed only because an rg in the same command happened to show the other session's text. It re-applied the edit with an assert (L197). The concurrency was disclosed. The silent no-op was not.
  3. Runs with non-Pi workers get no advisory audit (I5, 49178f22 L185/L212). The parcha-recall analysis ran with Opus workers. Its advisory audit was skipped because the meta-analyser supports only Pi, so none of its 18 worker attempts was scanned. The run also recovered from five first-pass verification rejections. All of this was disclosed.

Decision this supports: - Recommend a design-pressure diagnosis for I5. The audit gap affects every analysis run that uses Claude Code workers, and a Claude Code scanner now exists. - Collect one more observation for I2 before diagnosing. - I1 is a candidate for diagnosis only if same-checkout concurrent edits recur.

Coverage

  • Project: cwd /home/zby/llm/commonplace. Project directory: ~/.claude/projects/-home-zby-llm-commonplace/ only.
  • Interval: 2026-11-05T00:00Z to the live-file cutoff.
  • Selection: first and last record timestamps, as agreed with the operator.
  • Excluded: other files in this directory whose mtimes are recent only because untimestamped cost records were appended. I did not reopen them; the exclusion follows the operator's prior determination.
  • Selected sessions (scanner completed-tool counts):
  • 25868e95-d6c5-4168-af25-ff2b1fac1682.jsonl: 08:54Z 11-05 to 06:56Z 11-06, 929 lines, 72 tools, 1 compaction.
  • 49178f22-f588-405f-9442-ab30ea5f2c58.jsonl: 08:55Z to 15:23Z 11-05, 245 lines, 21 tools.
    • Worker subagents/agent-af36508657db43123.jsonl (forked cp-skill-ingest, depth 1): 119 lines, 16 tools. It launched:
    • agent-adbfb43558858a45b.jsonl (forked cp-skill-connect, depth 2): 109 lines, 24 tools.
    • agent-adb7d5119df78c63e.jsonl (drafting worker, depth 2): 33 lines, 4 tools, interrupted.
    • All three worker files are present.
  • 86f2287c-3b31-47ba-8c51-d3309dbdc370.jsonl: 16:45Z to 16:50Z 11-05, 6 tools.
  • 90e9e314-752b-4764-ab52-a59b1a2165ac.jsonl: 18:12Z to 18:17Z 11-06, 3 tools.
  • 3929c733-2dfa-4ee3-8405-dfdd1afe7c0d.jsonl (live): cutoff at line 304, last timestamp 2026-11-07T09:28:24.931Z, 21 tools to the cutoff. Later lines are out of scope.
  • Children excluded:
  • 3929c733/subagents/agent-ab3b2bdf821c459c7 is this review itself.
  • The parcha-recall analysis workers ran in worktree .commonplace/worktrees/parcha-recall-d5ad184aa4af. Their traces live outside the authorized project directory and were not examined.
  • The bootstrap-exploration workers referenced from 25868e95 (L823) are Pi sessions under ~/.pi and are out of scope.
  • No missing child files were found for in-scope launches.
  • Scanner: scan_claude_code_tool_failures.py exited 0, with no input issues. It reported 8 failure leads, 1 compaction, 1 interruption, 0 API errors and 4 worker IDs. All 8 leads, the compaction, the interruption and every worker launch were inspected.
  • Successful sequences sampled (12), favoring pipelines and output-dependent conclusions:
  • 25868e95: L207→L215→L229; L546→L547→L560; L567→L568→L576; L824; persisted-output read-backs L120/132–134 and L784/785/800/802.
  • 49178f22: L164→L185→L212.
  • af365: L56→L74.
  • 90e9e314: L67→L68→L75.
  • 86f2287c: L47→L55.
  • 3929c733: L138→L139→L225; L173→L174; L197/198 and L213/214→L225.
  • Not examined:
  • About 40 other successful calls in 25868e95, mainly the workshop-inventory loops at L56–L104 and L236–L398.
  • Most calls inside the adbfb connect worker.
  • Whether the 25868e95 compaction summary was faithful in content; only post-compaction behavior was checked.
  • Write-scope note: I briefly wrote one intermediate file, scan.json, holding scanner output, to the scratchpad. I deleted it before finishing. The only remaining output is this report.
  • Limits: absence of API errors is not proof that no provider retries happened, because Claude Code does not persist retry attempts. A clean sample is not proof of healthy execution.

Incidents

I1. Silent no-op edit under a concurrent writer. Session 3929c733, call at L173, result at L174, recovery at L187–L198. - Intended operation: insert the Claude Code scanner text into scripts/scan_tools/README.md, replacing the "Only Pi is supported" paragraph. - Failure: the edit was an unguarded s.replace(...). A concurrent session had already changed the anchor text to "Pi and Codex have separate parsers". The replace matched nothing, the file was rewritten unchanged, and the command exited cleanly. The intended replacement text also said "Codex is not yet supported", which was already false. - Recovery: the agent saw "Codex" in incidental rg output and re-read both READMEs (L188). It re-applied the edit with assert old in s (L197, prints ok). - Disposition: recovered, and checked against the outcome. The committed README (590cc0685) contains the Claude Code section. - Effects: - Extra work: 2 calls (L188, L197), about 20 s. - Assurance: none lost after the recovery. - Output defect: none. - Visibility: the concurrent edit was disclosed in L225 ("Another session ... edited both READMEs concurrently ... I reconciled"). The silent no-op appears only in the tool trace. - Hypothesis (not established): two agents sharing one checkout, combined with unguarded scripted edits, makes edits lost or stale without any error. The same session's later edits used assert, and so did 25868e95's (L414, L424, L546).

I2. Exhaustive-sounding conclusion based on capped evidence. Session 25868e95, calls L206 and L207, result L215, conclusion L229. - Intended operation: find recorded prerequisites between open workshops, to answer the operator's prioritisation question at L161. - Failure: the prerequisite search was grep ... | cut -c1-260 | head -40, and its section of the output has exactly 40 lines. That probably hit the cap. L206 read only lines 1–45 of five machine-workshop READMEs. - Claim to the operator: "nothing recorded says the semantics work gates the machine. I checked every open workshop README for stated prerequisites and cross-links". - Recovery: none. Later turns (L233–L307) traced the workshops' origins but did not re-check prerequisites. - Disposition: unknown. - Effects: - Extra work: none. - Reduced assurance: the claim of exhaustive coverage of stated prerequisites lacks complete evidence. - Output defect: not demonstrated; the uncapped match count was not established. - Visibility: not found in inspected communications. L229 states full coverage without qualification, and the away_summary at L231 repeats the conclusion.

I3. Operator-stopped ingest with nested workers and leftovers. Session 49178f22, L55–L83, plus workers af365, adbfb and adb7d. - Intended operation: the cp-skill-ingest fork for the parcha recall skill. The operator stopped it as a mistake (L55, "sorry - stop it"). - Recovery: TaskStop on af365 (L71). The drafting worker adb7d recorded [Request interrupted by user] at 09:04:00.213Z (adb7d L33). Its only calls were reads (L17, L25, L26, L30); it made no Write. - The parent told the operator (L83): "I can't confirm that worker stopped ... a kb/sources/parcha-skills-recall.ingest.md could still appear". The worker trace resolves that uncertainty: it stopped and wrote nothing. As of this review, kb/sources/parcha-skills-recall.ingest.md does not exist. - Leftovers that still exist: kb/sources/.snapshots/parcha-skills-recall.md and kb/reports/cache/connect/sources/parcha-skills-recall.connect.md. The temp capture directory was removed. - Disposition: the stop was recovered and is verifiable in the trace. Leftover cleanup is unresolved: the L83 question to the operator got no answer before the conversation moved on. - Effects: - Extra work: about 6.5 minutes of worker execution and 44 worker tool calls were abandoned at the operator's request. This was operator-initiated, not a system failure. - Assurance and output: no effect. - Visibility: disclosed (L83). - Also observed: adb7d read only lines 40–400 of a 645-line persisted output (L30) before it was interrupted. This had no effect, because the worker stopped.

I4. Operator said "integrated it", but nothing was integrated. Session 49178f22, call L224, result L225 (exit 2), conclusion L243. - The nonzero exit was the evidence: ls of retained/parcha-ai--parcha-skills/ failed on main. The agent corroborated this with L232 and L238 and concluded that nothing had been integrated. - Present repo state is consistent: no commit mentions parcha, no parcha branch exists, and worktree parcha-recall-d5ad184aa4af is still at detached cbf2389b5. - Disposition: the check was correct; the integration itself is unresolved. - Effects: none on execution. - Visibility: disclosed (L243, away_summary L245).

I5. Advisory audit skipped for a Claude Code–worker analysis run. Session 49178f22, result L185, report L212. - The run log at L185 says "advisory audit: skipped; ... meta-analysis supports Pi only". - L212 reports: - the trace scan covered 18 attempts but scanned none of them; - the first verification pass rejected four reports and the memory profile, all accepted after correction. - Exit code 0 was captured explicitly with echo "exit $?", not masked. - Disposition: the run completed, but the audit requirement was not met. - Effects: - Reduced assurance: no execution-health audit evidence exists for 18 worker attempts. - Extra work: 5 rejected first-pass products. Their cost was not observable from the parent trace. - Output defect: none demonstrated. - Visibility: disclosed (L212, "Advisory audit: skipped ...").

I6. Stale test claim and no test run at commit. Session 3929c733: L198 ("124 passed"), L213 edits the scanner and its test, L214 ("16 passed"), claim at L225, commit at L249. - L225 says "General and tooling suites pass (124 tests)". That run preceded the L213 edit to scan_claude_code_tool_failures.py. After the edit, only the scanner's own 16 tests ran. No tests ran before the commit at L249. - File mtimes at L241 show no scanner or test changes after 08:55Z. - Disposition: recovered in effect. The reduction in assurance is minor. - Effects: - Reduced assurance: after the last code edit, the general and tooling suites were not re-run before the commit, which the repo rule asks for. - Output defect: none demonstrated. - Visibility: the claim was presented without qualification.

I7. xargs masked the scanner's exit code. Session 3929c733, L138, L139 and L225. - find | xargs -0 scanner > corpus.json; echo "exit $?" printed exit 123, which is xargs's generic code for a failed child. L225 reports "exited 2", which is an inference, not an observation. It is consistent with the two input issues shown in the output. - Disposition: recovered. - Effects: negligible. - Visibility: the reported exit code is slightly overstated as observed.

Compaction. Session 25868e95, L629. - An auto compaction happened while the session was idle, waiting on a ScheduleWakeup (L621, 3600 s): preTokens 204,423, logicalParent eeb1a9d3-…. L639 notes it happened "while idle, before the prompt cache expired". - After compaction, the wakeup loop resumed (L647 onward), and the review re-read every input from disk (L754–L824). No reliance on lost context was observed. - Disposition: recovered. Content fidelity was not checked. - Visibility: trace only. No assistant message mentions it.

Groups

Expected nonzero exits, no defect (4 leads). Each is a shell idiom or an intended absence check. - 25868e95 L67 (exit 1): the loop's last [ -gt 8 ] && echo was false. The output was complete, and L79 used all five hashes. - adbfb L95 (exit 2): ls kb/sources/parcha* found nothing, with stderr hidden. - af365 L63 (exit 2): the same snapshot-absence check. - af365 L109 (exit 2): ingest-absence check before dispatching the drafting worker.

Scanner false positives, a scanner precision issue rather than an execution incident (3 leads, all in 3929c733). - L78, L91 and L148 have is_error: false. They were flagged tool_use_error or permission_denied because their output quoted trace text. - Cause: the scanner (scan_claude_code_tool_failures.py, roughly lines 107–112) applies its substring signals without checking is_error. - Any session that reads logs will produce these. The authoring session saw the same two records in its corpus run (L139) and did not change the classification.

Silent-success edits and capped evidence (2 incidents, 2 sessions). I1 is an unguarded replace. I2 is a head-capped search with a coverage claim. Both are masking mechanisms that leave a successful exit. Guarded edits were the norm elsewhere (25868e95 L414, L424, L546; 3929c733 L197, L213).

Persisted-output overflow read-back (routine, 2 sessions). - 25868e95: L120, L784 and L785 overflowed again and had to be re-read. That is 3 extra calls, about 10 s. Full coverage was eventually reached for all persisted files (331, 350 and 414 lines). - adb7d: a partial read of 645 lines (I3), with no effect.

Disclosed, operator-dependent follow-ups (I3, I4, I5). All were visible in final replies.

Clean samples (no finding): - 25868e95: L546 guarded edit plus validate; L567 commit with stat; L824 validate | tail -8 showing VALIDATION SUCCESS. - af365 L56: gh api loop with || echo FAIL, verified by hashes at L74. - 90e9e314 L67: an empty backlink result with no stderr, which supports the "orphan" claim at L75. The same command counted tags. - 86f2287c L47: the uv run import printed a path. - 49178f22 L164: explicit exit capture.

Follow-up

  • Recommend a design-pressure diagnosis (Review design pressure behind recurring errors, /home/zby/llm/commonplace/kb/instructions/review-design-pressure-behind-recurring-errors.md) for I5. Hand off:
  • L185 and L212;
  • the advisory-audit path that rejects non-Pi traces;
  • the now-committed Claude Code scanner (590cc0685).

The question for the diagnosis: should the analysis audit accept Claude Code worker traces? The traces sit in a worktree project directory outside this review's scope. Diagnosing them needs authority over that directory. - More evidence needed, I2. The smallest useful observation: count prerequisite-pattern matches in kb/work/*/README.md as of commit cbf2389b5, with no cap. If the count exceeds 40, check whether any of the extra lines records a semantics→machine dependency. If none does, close I2 as no defect. Visibility was not found either way. - Diagnosis only if recurrence, I1. Watch for another silent no-op or stale edit while two agents work in the same checkout. - Scanner precision. Gate the substring signals on is_error, or report them separately. This is a scanner change for the operator to commission. This review does not authorize it. - No further action: I4 (correct check, disclosed), I6 and I7 (minor, no defect), the compaction, and the expected-exit group. - Operator housekeeping, not a defect: I3's leftover snapshot and connect report await the operator's keep-or-delete decision. The parcha analysis awaits integration.

Verify checklist

  • Recovered incidents remain visible: yes (I1, I3, I4, I6, I7, compaction).
  • Findings have trace evidence, and causes are qualified: yes. The cause of I1 is labelled a hypothesis. I2's truncation is inferred from the exact cap and is not proven.
  • Counts exclude duplicate representations and identify branch scope: yes. Each tool call was counted once by tool_use_id. Parent-link continuations were followed for each lead. No rewound branches were encountered in the inspected leads; I did not enumerate rewinds exhaustively across all files.
  • Operator visibility is assessed separately and cites a communication: yes.
  • Missing traces and the clean sample are not treated as proof of health: yes. The out-of-scope worktree and Pi worker traces are listed as unexamined.
  • Supports a bounded diagnosis handoff without modifying the system: yes.
  • Not fully satisfied:
  • Compaction-summary fidelity was not checked.
  • About 40 successful calls in 25868e95 and most of the adbfb worker were not sampled.
  • The temporary scratch file was a brief write-scope deviation, since deleted.