CrewAI report verification

Type: agentic-system-analyses/types/agentic-system-verification.md

Verification

The current reports support a bounded implementation account. The previous feedback-selector blocker is repaired in the route and its dependent passages. No material blocker remains. The structural report-check says none; there is no structural finding to route.

I read every supplied member, declaration, annotation, source attribution, containment relation and amendment, the correction answer and the previous verification. I judged their meaning afresh rather than treating the earlier judgment or analyst checks as evidence. Independent inspection used SRC-1 at the registered boundary. HEAD matched the frozen revision, status was clean and the tracked top-level tree matched the boundary inventory. Source reads and searches establish implementation or attributed doctrine/design only. No CrewAI execution, tests, examples, provider calls or causal experiment were performed. Deterministic quotation occurrence and link/form checking are not semantic verification.

Runtime and workflow

  • RT-CMP-agent-llm, RT-CMP-memory-llm and RT-CMP-memory-embedder distinguish model bindings from retained outputs and parameter updates. SRC-1, lib/crewai/src/crewai/utilities/llm_utils.py and lib/crewai/src/crewai/memory/unified_memory.py support configurable string/instance/dictionary resolution, separate memory analysis and the default embedding specification. The evaluator binding and training account do not establish an independent oracle. Exact provider weights, endpoint stability and provider-side updates remain outside the inspected implementation.
  • RT-RTE-crew-kickoff, RT-RTE-agent-delegation and the workflow account preserve separate scheduling and action owners. SRC-1, lib/crewai/src/crewai/crew.py and lib/crewai/src/crewai/tools/agent_tools/base_agent_tools.py support ordered task scheduling, manager execution, bounded coworker selection and selected task context. The helper directly invokes coworker agent execution; constructing a Task does not run its outer guardrail/callback/file lifecycle. The workflow selects a connected conditional branch, not evidence that delegation happened. Last-truthy-output aggregation and later hooks support API delivery, not an independently certified artifact or external receipt.
  • RT-RTE-tool-dispatch and RT-CLM-tool-veto are supported as local protocols. Independent reads of SRC-1, lib/crewai/src/crewai/agents/crew_agent_executor.py, lib/crewai/src/crewai/experimental/agent_executor.py, lib/crewai/src/crewai/utilities/tool_utils.py and lib/crewai/src/crewai/hooks/tool_hooks.py establish blocking before function invocation on the named native and synchronous/asynchronous ReAct helpers. Post-call replacement changes observations. These helpers neither sandbox trusted Python nor mediate all direct calls.
  • RT-RTE-output-guardrail, RT-CLM-output-validation and RT-RTE-agent-kickoff retain the consequential ordering distinction. SRC-1, lib/crewai/src/crewai/task.py, lib/crewai/src/crewai/agents/agent_builder/base_agent_executor.py, lib/crewai/src/crewai/agents/crew_agent_executor.py and lib/crewai/src/crewai/agent/core.py support Task checking after executor memory submission, but standalone finalization checking before its kickoff save. The inherited executor save requires a task; a save call name alone does not establish another unconditional standalone write. Retry receives validation-error context and can repeat effects. Ordered checks do not rerun all earlier checks after later replacements; post-step hooks follow checking. The workflow annotations preserve these limits instead of claiming transactional acceptance.
  • RT-RTE-flow-execution is supported by SRC-1, lib/crewai/src/crewai/flow/runtime/__init__.py: arbitrary Python method invocation, hook-edited parameters, UUID hydration and checkpoint-specific completed-method state are distinct responsibilities. Thread scheduling is not privilege isolation. UUID hydration alone need not skip completed work, and input fields can override hydrated state.
  • RT-RTE-context-summary and its memory annotation accurately describe retained working context. SRC-1, lib/crewai/src/crewai/utilities/agent_utils.py preserves system messages and files while replacing non-system history with model summaries for the next invocation. Neither losslessness nor automatic verbatim recovery follows.
  • RT-OBJ-skill, RT-RTE-skill-disclosure and RT-BAP-skill-instructions distinguish configured instruction content, catalog selection and model-context delivery. SRC-1, lib/crewai/src/crewai/skills/tool.py and lib/crewai/src/crewai/skills/loader.py support disclosure rather than automatic skill writing or enforced compliance.
  • RT-OBJ-trained-agents, RT-RTE-crew-training and RT-BAP-training-instructions correctly separate raw feedback records, evaluated suggestions, pickle loading and later role-keyed imperative instructions. SRC-1, lib/crewai/src/crewai/crew.py, lib/crewai/src/crewai/utilities/evaluators/task_evaluator.py, lib/crewai/src/crewai/utilities/file_handler.py and lib/crewai/src/crewai/agent/core.py support the producer-to-consumer chain. Human task feedback is not approval of each suggestion. The memory annotations add no stronger operation, weight-training or improvement status.

Memory and epistemic findings

  • MEM-OBJ-memory-content, MEM-OBJ-memory-metadata, MEM-OBJ-memory-vector, EPI-OBJ-memory-record, EPI-OBJ-consolidation-plan and EPI-OBJ-memory-match separate stored text, selection/provenance fields, numerical outputs, transient plans and result wrappers. SRC-1, lib/crewai/src/crewai/memory/types.py, lib/crewai/src/crewai/memory/analyze.py, lib/crewai/src/crewai/memory/encoding_flow.py and lib/crewai/src/crewai/memory/storage/lancedb_storage.py support these identities. Storage and source labels are not authenticated warrant; database reopening establishes wiring, not universal durability.
  • MEM-RTE-memory-encoding, EPI-RTE-memory-extraction, EPI-RTE-consolidation-check and EPI-RTE-plan-application retain distinct transformation, checking and application functions. SRC-1, lib/crewai/src/crewai/agents/agent_builder/base_agent_executor.py, lib/crewai/src/crewai/memory/unified_memory.py, lib/crewai/src/crewai/memory/analyze.py and lib/crewai/src/crewai/memory/encoding_flow.py support extraction, explicit-write alternatives, serialized/background saves, embedding, similarity handling and model-plan application. Full-content extraction fallback, default insertion on consolidation failure, fast insertion and truncated old-content comparison defeat stronger factual-acceptance interpretations. The epistemic ledger's indeterminate generated-content relations are appropriately distinct from observed conjecture or demonstrated semantic preservation.
  • MEM-CLM-retention-limits is substantive: reconstruction omits source/private, defaults change those fields, and LanceDB serializes the reconstructed record through delete/reinsert update. SRC-1, lib/crewai/src/crewai/memory/encoding_flow.py, lib/crewai/src/crewai/memory/types.py and lib/crewai/src/crewai/memory/storage/lancedb_storage.py support the counterexample. This is not an observed leak. Transient plan reasons are not automatically persisted, and comments do not establish an encompassing transaction.
  • MEM-RTE-memory-recall, EPI-RTE-memory-selection and MEM-BAP-memory-context accurately distinguish requests, selectors and delivery. SRC-1, lib/crewai/src/crewai/memory/recall_flow.py, lib/crewai/src/crewai/memory/unified_memory.py, lib/crewai/src/crewai/memory/memory_scope.py, lib/crewai/src/crewai/tools/memory_tools.py and lib/crewai/src/crewai/agent/core.py support caller limits, scope/slice merging, write barriers and composite relevance ranking. Re-search repeats selector inputs; final synthesis ranks records. Formatting omits explicit gaps, source and match reasons even when full objects retain them. The epistemic formatting example is deterministic code reasoning, not target execution. Neither relevance scores nor suggested scopes establish truth or strict security isolation.
  • MEM-RTE-memory-maintenance and EPI-RTE-memory-mutation preserve the direct API distinction between update's read-only check and forget/reset bodies without that check. SRC-1, lib/crewai/src/crewai/memory/unified_memory.py also supports retained-vector fallback on failed re-embedding. These are bounded API findings, not proof about all wrappers, backup erasure or propagation to prior prompts.
  • MEM-OBJ-knowledge-chunks, MEM-RTE-knowledge-context and EPI-RTE-knowledge-import preserve acquisition rather than produced warranted knowledge. SRC-1, lib/crewai/src/crewai/knowledge/knowledge.py, lib/crewai/src/crewai/knowledge/source/base_knowledge_source.py, lib/crewai/src/crewai/knowledge/storage/knowledge_storage.py and lib/crewai/src/crewai/agent/utils.py support configured source admission, slicing, content-only storage documents and later task-context delivery. The reports do not promote this witness into parser-wide fidelity or provenance.
  • MEM-OBJ-flow-state, MEM-OBJ-runtime-state, MEM-RTE-output-replay and MEM-RTE-checkpoint-restore distinguish snapshots, pending review context, entity/event checkpoints and reconstructed prior TaskOutputs. SRC-1, lib/crewai/src/crewai/flow/persistence/sqlite.py, lib/crewai/src/crewai/state/runtime.py, lib/crewai/src/crewai/state/provider/json_provider.py, lib/crewai/src/crewai/state/checkpoint_listener.py and lib/crewai/src/crewai/crew.py support reconstruction consumers rather than semantic retrieval or truth acceptance. Payload fidelity and alternative-provider uncertainty remain local.
  • EPI-RTE-test-scoring, EPI-RTE-eval-gate, EPI-CLM-memory-facts and EPI-CLM-eval-exit preserve reporting, local enforcement and doctrine separately. SRC-1, lib/crewai/src/crewai/utilities/evaluators/crew_evaluator_handler.py, lib/cli/src/crewai_cli/experimental/eval_crew.py, lib/cli/src/crewai_cli/cli.py, docs/edge/en/concepts/memory.mdx and lib/crewai/src/crewai/translations/en.json support those distinctions. Model quality scores and fact terminology do not establish an answer oracle. The remote done/passed predicate controls local process status, not proven remote judgment correctness or deployed CI behavior.

All route wired implementation and uninspected operation statuses remain warranted. No evidenced-absence record is declared. The narrow lifecycle-integration search is not a framework-wide negative. Retention and training labels establish none of the independently unestablished learning, reflection, autonomy or theory-builder properties; missing runs alone are not used to negate architectural capacity.

Correction and reconciliation

The epistemic answer is supported by fresh inspection of SRC-1, lib/crewai/src/crewai/flow/human_feedback.py, lib/crewai/src/crewai/memory/unified_memory.py and lib/crewai/src/crewai/memory/recall_flow.py. Pre-review requests default-deep recall with learn_source, but both depth paths admit public records from other sources. The reviewer receives content strings labelled as lessons, without source fields. EPI-RTE-feedback-lessons, EPI-OBJ-feedback-lessons, the authority-route ledger, coverage and bounded conclusion now preserve that distinction. The answer repairs the dependent findings, not just a phrase. MEM-RTE-feedback-lessons, MEM-BAP-feedback-review and the epistemic route still support distillation, background retention and compatible-output rewriting, not exclusive feedback lineage or reliable improvement. Their retained passages supply the selector and consumer evidence. The earlier blocker is discharged, not converted into a limit.

The reconciliation correctly removes the obsolete disagreement. No current unresolved cross-report conflict remains. Its two amendments preserve identity: the content objects denote MemoryRecord.content; the feedback routes denote the same distillation/pre-review chain. Supersession transfers no stronger warrant or operation status. The narrower epistemic extraction, consolidation, application, selection, mutation and import routes remain useful parts of valid broader memory accounts, not whole-record duplicates. The feedback-to-Flow and replay-to-Crew Part of: relations are supported; cross-runtime checkpoint restoration is not wholly contained in one runtime route. Distilled lesson payload, match wrapper, plan, vector, Flow snapshot and runtime checkpoint remain distinct referents. Workflow annotations add no competing declarations. Memory integration issues are disposed without erasing supported counterexamples or unresolved alternatives.

Omission questions and stopping decision

I selected omission questions from the frozen tree, alternate execution owners and retained-state producer/consumer dependencies, not just listed anchors.

  1. Could alternate execution invalidate the local tool-veto, output-gate or delivery account? Native Crew and standalone hook blocks, both ReAct helpers, Task retry/post-step ordering, standalone finalization and direct Flow dispatch preserve the named guarantees and their limits. The task precondition on inherited saving prevents inventing an unconditional extra standalone retention route.
  2. Could another admission or consumer change the retained-context conclusions? Searches for extraction, batch saves and recall across shipped core, CLI and scripts located Flow convenience APIs, memory tools, TUI recall and deprecated LiteAgent. Convenience APIs do not alone establish additional deployed consumers. LiteAgent does supply another concrete last-user-input consumer with content-only system-message delivery; it remains an inventory limit below. Fresh deep/shallow selector tracing resolves the central source-exclusivity challenge. Fast paths, fallback retention, formatter omissions and the backend update consumer preserve the other findings.
  3. Could recovery or evaluation be mistaken for accepted knowledge or safe replay? Replay reconstruction, UUID/pending-feedback SQLite state, checkpoint restoration, model-score consumers and local remote-verdict status handling support the reports' distinctions. Uninspected backend/provider payload alternatives prevent stronger fidelity, erasure or reliability comparisons, not these positive routes.

Further optional expansion stopped because every identified consequential challenge has a supported disposition, the former contradiction is repaired, and remaining alternatives concern explicitly withheld completeness, backend or empirical conclusions. Additional adapters and archived guides alone do not warrant an exhaustive audit. Unsearched behavior is not evidenced absence.

Blockers

none

Limits

  • MEM-RTE-memory-recall and MEM-RTE-memory-encoding establish positive Crew/Agent/tool routes, not every shipped caller or delivery form. Independent source inspection confirms deprecated LiteAgent's last-user-input recall, content-only system-message injection and extracted-output save in SRC-1, lib/crewai/src/crewai/lite_agent.py. This does not change the named modern Agent budgets or central extraction/encoding mechanism. Withhold exhaustive caller counts and universal formatted-score/metadata delivery comparisons; file-cache and adapter-specific consumers likewise remain outside the detailed inventory. The additional caller's different delivery is explicit:

    matches = self._memory.recall(query, limit=10) if matches: memory_block = "Relevant memories:\n" + "\n".join( f"- {m.record.content}" for m in matches ) --- lib/crewai/src/crewai/lite_agent.py

  • MEM-OBJ-memory-content, MEM-RTE-memory-maintenance, MEM-RTE-knowledge-context, MEM-OBJ-flow-state and MEM-RTE-checkpoint-restore retain unresolved alternate-backend merge/deletion, non-string preprocessing, custom persistence, pending-feedback continuation and opaque checkpoint payload fidelity. Positive inspected routes remain supported. Withhold complete cross-process durability, global withdrawal/erasure, parser-wide provenance, exact hidden-payload replay and exactly-once recovery comparisons; source inventories and nominal schemas do not settle them.