OS-Copilot synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
From request to host action
OS-Copilot ships an enclosing runtime for computer-operating agents. Its principal agent, FRIDAY, owns model-directed planning, sequential execution, judgment, repair and replanning. The evidence is source inspection at commit f720af8807e49a92dda64572d2c6bc6c0ac7ee7e, with an applicability cutoff of 2026-10-16. SRC-1 supplies implementation, design instructions and a tutorial's attributed learning outcome, but no inspectable execution trace or causal experiment. The whole-system boundary includes the task and course drivers, UI, tool services, retained-tool writers and consumers, examples and evaluation code. External model services, dependencies and the live operating environment are not authenticated by this inspection. The account establishes connected implementation paths, not successful deployment (boundary, Source register).
The shipped quick_start.py caller constructs a planner, retriever, executor, tool manager and FridayAgent, then supplies task text. Its default request copies text files containing “agent” into an “agents” directory. This is a representative connected workflow, not an observed sequence of discovery, reading and copying operations. The UI launches the same script and interprets process/log output; it is not a separate agent controller or correctness oracle (RT-RTE-task-execution, workflow account).
FRIDAY first retrieves up to ten retained-tool descriptions using task-text similarity. The planner model receives those descriptions, the request, OS information, directory listing and API descriptions. It proposes a JSON dependency graph. Fixed code topologically orders incomplete nodes, and FridayAgent executes them one at a time. These are in-process responsibility handoffs, not independently authenticated workers or parallel agents. Direct prerequisite descriptions and return values cross to each successor; predecessor histories do not automatically cross. Python generation separately retrieves up to three retained code examples (RT-OBJ-tool-graph, RT-BAP-tool-context, workflow annotations).
The model proposes a program and, for Python, an invocation. The executor appends result-capture code and dispatches to Jupyter, shell or AppleScript through Env. Host credentials, environment variables, installed packages and applications determine the effective capability surface. The inspected dispatch chain has no intervening per-program human approval. A configured working directory supplies context, not demonstrated filesystem confinement. Ordinary Python-step kernel teardown removes interpreter state, not files or other host effects (RT-RTE-task-execution).
Context is assembled afresh for the main model calls as one system and one user message. Selected tools, predecessor results and explicit critique/state fields replace automatic full-transcript carryover. Graph and inner-monologue state reset on the next run. This keeps call inputs selective, but it is not delegated-agent isolation: no independent worker frame returns a product. Stored tools have an explicit reload route; discarded full conversational history has no analogous established reload (RT-RTE-task-execution, memory annotations).
Feedback changes continuation, not every output's warrant
For Python, Shell and AppleScript nodes, a model judge consumes code, task requirements, errors, directory information and successor requirements. Long output is reduced to its first 999 characters when its length exceeds 1000; repair receives the full output. The resulting status and score have consequential consumers: Amend triggers revision and re-execution, and Replan inserts prerequisites. The initial branch treats other statuses as completion, whereas successful repair requires Complete. Thus a completion flag is neither a uniform positive verdict nor an independent check of the resulting world state. The judge's generality score also extends beyond the evidence of a single task execution (RT-RTE-feedback-repair, EPI-OBJ-judgment-result, EPI-CLM-code-judgment).
Repair has a local attempt ceiling, not a whole-task termination bound. Replanning can repeatedly add work, and neither path rolls back prior host effects. Language-specific repair remains only an attempted mechanism: Shell/AppleScript prompts feed shared Python-fence extraction and unconditional invocation-marker indexing, so a missing marker can fail before execution. Queue exhaustion is also weaker than graph completion because planner callers do not enforce the cycle diagnostic. run returns no explicit final success/failure product. Filesystem effects and printed/logged results are the delivery surfaces; its accessible last accepted result need not summarize or verify the whole task (RT-RTE-feedback-repair, workflow recovery, workflow delivery, report verification limits).
Other node types and shipped loops materially change this policy. QA prints a model answer; API generates Python dispatched through the Python environment's API alias without the ordinary Python invocation wrapper. Both bypass the code judge. LightFriday and the vision example instead accumulate histories, execute the first fenced code block and stop on a response substring. Their existence prevents applying FRIDAY's judge, retention policy or two-message context policy to every shipped invocation (RT-RTE-api-qa, RT-RTE-example-loops).
External information has separate warrant limits. The mounted page service cleans HTML/PDF text, returns full extracted text at no more than 4096 counted tokens, and otherwise chooses a map-reduce summary without a query or three similarity-selected chunks with a query. Selection does not impose a final response token cap or corroborate claims. Loader metadata is not forwarded with the text, exceptions can become empty content, and punctuation cleaning can alter formal expressions. The verifier qualifies the report's illustration: changing 1 == 1 to 1 = 1 produces invalid Python, not a valid assignment. The character-change deduction still defeats universal semantic-preservation claims. Actual summary fidelity, source truth and downstream uptake remain unobserved. QA, caption, audio and OCR outputs likewise have no demonstrated independent factual acceptance gate in these routes (EPI-RTE-page-content, EPI-OBJ-qa-answer, RT-RTE-api-qa, report verification limits).
From task feedback to durable resources
The strongest supported learning-related contribution is a wired automatic path from execution feedback to durable reusable artifacts and later model-context delivery. A completed Python node meeting the configured generality threshold can be stored under a new name. Local JSON and mirrored code/description files retain the payload; persistent Chroma embeds descriptions and ranks tool names. New manager instances reload state, and later planning/generation retrieves descriptions or code. This is a feedback-responsive, behavior-shaping write/read chain, not merely raw-log accumulation. It supports wired trace learning from tool execution feedback in the comparison's write-route sense; improved capacity is a separate, unestablished conclusion (RT-RTE-tool-admission, MEM-OBJ-tool-code, MEM-OBJ-tool-vectordb, profile verification).
The chain has a candidate-identity defect: successful repair replaces the outer score and result but not the outer code subsequently offered to storage. Original failed code can therefore be admitted using a repaired candidate's verdict. The store also retains code and a docstring fragment rather than structured task, critique, score or outcome lineage. Future retrieval ranks relevance, not correctness. It delivers examples to a model that generates fresh programs; it does not automatically invoke stored functions (RT-RTE-tool-admission, MEM-OBJ-tool-description, RT-BAP-tool-context).
Manual maintenance reaches the same stores without the agent's score/execution gate. A documented human can supply or edit a selected file, but generic manager callers do not establish per-content human approval. Overwrite revises entries; deletion removes persistent payloads/index entries while leaving stale live dictionary copies. Later operations can encounter or reserialize those copies. Count assertions establish neither content agreement nor atomic recovery. Consequently automatic admission, manual content supply, generic API writes and withdrawal retain different guarantees (RT-RTE-manual-maintenance, memory annotations).
Course learning supplies another progression: model-designed lessons become ordinary FRIDAY requests, while saved lesson text guides later curriculum design. Initialization loads a software/package-selected file; later design automatically receives up to the last fifty dictionary entries. This is a coarse count/order window, not a token budget or success-based recency policy. Continuous mode updates memory before attempting lessons, and saving consumes no verified lesson-success signal. The prompt nevertheless frames prior courses as completed. This is durable curriculum and task selection, not a mastery record. Optional demo extraction reuses the agent's last planner-node result rather than independently certifying extracted content (RT-RTE-course-learning, MEM-OBJ-course, RT-BAP-course-tasks).
The benchmark's saved rows have a narrower later consumer: accounting and resume selection, not tool admission or agent learning. GAIA compares a model-postprocessed answer with a reference by exact string equality. This establishes equality of operands, not semantic answer correctness or the quality of the producing process. The single-item branch's literal answer and the lack of supplied result traces prevent performance inference (EPI-RTE-gaia-comparison, report verification).
Independent property assessments and what would change them
For FRIDAY plus its retained tool state, the objective is future reusable computer-operating capability and the horizon is later tasks/sessions. Feedback-controlled admission, durable update and a live context-delivery path support an improvement-directed dispositional pathway. Subsequent operation causally depending on the admitted change—the fourth self-improvement link—is unestablished. Exercised self-improvement and successful improvement therefore remain unobserved; the tutorial reports an Excel before/after outcome but supplies neither inspectable dependent-use evidence nor causal attribution. Learning as improved future capacity is likewise unestablished. Neither uncertainty removes the implemented trace-fed write/read contribution (RT-RTE-tool-admission, EPI-CLM-excel-learning).
Theory-builder conditions remain separately unestablished at this boundary: (1) no concrete generated candidate establishes localized formulated explanatory content; (2) no trace establishes decisions depending on such content; (3) generic judging and code repair do not establish criticism naming a theory claim and placement of blame; (4) keeping a revised program for another attempt does not establish a retained result of theory-directed criticism. Criticism of a consumed theory improving future capacity is therefore unestablished, independently of feedback-responsive program repair. The shipped method prompts are not revised by that loop. Reflection is not established by the supplied records' program judgments: they do not establish the required self-representation and both causal directions. Computational selection and execution without per-action approval are wired in the ordinary task route; autonomous theory building is not established because the internal theory-building operations themselves are not established. These are independent findings, not a single negative grade (RT-RTE-feedback-repair, RT-OBJ-friday-prompts, epistemic conclusion).
For a request such as the shipped file-copy task, the source supports an agent-owned plan/execute/recover progression but not a verified delivery acknowledgment. For repeated software practice, it supports persistent resources and curriculum context but not demonstrated mastery or cross-task reliability. A trace linking task, original/repaired candidate, verdict, stored identity, later retrieval and behavior would establish substantially more about activation. Matched capacity/outcome comparisons with independently varied retained state would address benefit and attribution at that comparison's scope. System changes that bind storage to the evaluated candidate, retain decision lineage, check whole-task outcomes and enforce graph/status validity would remove specific implementation obstacles, but would not substitute for outcome evidence. Concrete theory statements and claim-directed criticisms carried into subsequent decisions would be needed to reassess the theory-builder properties (RT-RTE-tool-admission, workflow account, epistemic conclusion).
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No inspectable run, later dependent-use trace or causal comparison | SRC-1; RT-RTE-task-execution, RT-RTE-tool-admission, EPI-CLM-excel-learning | Frozen repository implementation and attributed tutorial outcome | Actual task success, activation, improved capacity, exercised or successful self-improvement and causal component benefit | Provenanced execution and later-use traces; outcome comparisons with intervention and attribution controls |
| Repaired verdict can authorize original code; admission lineage is not retained structurally | SRC-1; RT-RTE-tool-admission, MEM-OBJ-tool-code | Automatic Python admission and manager payload | That retained code is the successfully evaluated candidate or carries its warrant | Identity-linked candidate/verdict/storage records; corrected storage data flow and confirming traces |
| Delivery and recovery controls are local and branch-specific | SRC-1; RT-RTE-feedback-repair, RT-RTE-api-qa, RT-OBJ-tool-graph, workflow annotations | Ordinary task graph, judgment, alternate node completion and queue termination | Uniform correctness gate, global termination, rollback or normal-return-as-success | Enforced graph/status checks, explicit whole-task delivery criteria and outcome evidence across branches |
| Language-specific repair parsing is coarsely represented in the reports | SRC-1; RT-RTE-feedback-repair, workflow recovery, report verification limits | Shell/AppleScript prompt selection followed by shared Python-fence extraction and invocation-marker indexing | Equivalent successful repair handling across languages | Language-specific candidate/parse/dispatch traces and parser contracts covering missing markers |
| Page-cleaning illustration mislabels invalid Python as assignment | SRC-1; EPI-RTE-page-content, report verification limits | Static repeated-punctuation transformation | The specific claim that 1 = 1 is a valid Python assignment; not the supported character-change or semantic-preservation concern |
Correct syntax analysis or a valid domain-specific illustration; actual content pairs for fidelity assessment |
| Provider/dependency state and page-client uptake remain unresolved | SRC-1; RT-CMP-chat-model, RT-CMP-tool-embeddings, EPI-RTE-page-content | Shipped adapters and page-response path, not external implementations | Exact model fixity, provider retention, page-cache lifecycle/isolation, faithful summaries or actual later-model consumption | Pinned provider/dependency contracts and state, generated caller traces and content-fidelity evaluation |
| Initial/direct-caller provenance, alternative helper consumers and withdrawal are incomplete | SRC-1; MEM-OBJ-tool-code, MEM-OBJ-tool-description, RT-RTE-manual-maintenance, RT-BAP-tool-context | Loaders, direct manager APIs, scoped helper searches and stale live dictionary | Complete memory inventory, universal human approval, all later-consumer routes or complete revocation | Initial-content provenance, caller tracing and coherent live/disk/index deletion/recovery evidence |
| Saved curriculum is not checked mastery | SRC-1; MEM-OBJ-course, RT-RTE-course-learning | Lesson design, dispatch, dictionary selection and save/merge | That prior lessons succeeded or increasing curriculum difficulty demonstrates learning | Lesson-linked checked outcomes, success-conditioned progression and capacity comparisons |
| Benchmark and theory assessments have narrow evidence scopes | SRC-1; EPI-RTE-gaia-comparison, RT-RTE-feedback-repair | Exact-string accounting and unidentified generated program content | Benchmark accuracy or process quality; theory building, theory-directed learning, reflection or autonomous theory building | Authenticated benchmark runs with reference/postprocessor checks; concrete theory/criticism/decision traces and property-specific causal evidence |
The report verification's two declared limits are carried above. Profile verification declares no additional limits; its partial inventory does not establish absent alternatives. Reconciliation identifies no unresolved cross-report conflict. No additional source reads were made for this synthesis.