Self-Training-LLM synthesis
Type: agentic-system-analyses/types/agentic-system-synthesis.md
Bounded synthesis
An improvement plane with a later model consumer
Self-Training-LLM's strongest supported contribution is a wired path from retained generated answers and scoring feedback to durable parameter updates and later inference. It is more than dataset storage: software selects training examples, invokes SFT or DPO optimization, saves parameters and provides a later checkpoint loader. Whether that path was exercised, changed subsequent answers or improved capacity remains uninspected. No executed comparison or intervention was supplied, so neither an overall gain nor a component effect is established (RT-RTE-model-training, RT-BAP-checkpoint-consumption).
The evidence is source code and attributed design documentation from SRC-1, https://github.com/wj210/Self-Training-LLM, at commit 97839b29d0fd8bb474f5549fa3e9d6ca504732e0, with an applicability cutoff of 2026-10-15. The whole-system boundary covers the shipped builder workflow, including data producers, retained-state consumers, training, serving configuration and evaluation. It does not cover external provider, dataset or trainer internals. No generated dataset, weights, provider transcript, evaluation output or execution trace was supplied. This synthesis uses the accepted member records; it adds no target-source reads (boundary and evidence, Source register).
This is not a conversational agent that decides which stage to execute next. An operator separately launches generation, training, serving and evaluation. Python owns stage-local sequencing; language models propose content or judgments; the NLI classifier supplies numerical estimates. Models do not direct shell execution. The runtime map is generation → persisted candidates/scores → training → checkpoint → separately selected inference → saved responses → pairwise judgment and aggregate report. Files, rather than live delegated conversations, cross stage boundaries (RT-RTE-data-generation, workflow account).
Construct candidates with deliberately unequal context
The selected workflow is a Wikipedia DPO experiment using an SFT model to produce preference candidates, then training a DPO checkpoint and comparing its held-out responses against SFT responses. Document acquisition retains article text in a title-keyed pickle. Predefined topics select documents; cleanup removes reference tails, and chunking selects bounded text. These operations import and reshape assertions without certifying their truth or preserving complete source qualifications (MEM-OBJ-topic-docu, EPI-RTE-document-transform).
Question generation receives document chunks and sampled shipped examples. DPO reference answers can receive documents, while raw samples omit them. Ordinary self-generated SFT labels and optional external labels have different premises from the auxiliary document-conditioned SFT stream. Thus gold_answer names an optimization role, not a uniformly grounded or independently certified answer. Lexical checks, retries, cleanup and omission admit candidates; they do not accept factual assertions against independent evidence. No supplied output/premise pair establishes whether generated content repeats, entails or adds a claim (EPI-OBJ-gold-answer, EPI-RTE-answer-cleaning).
Working context is constructed per call. Only selected inputs enter each thread task, and returned content escapes it. Fixed examples, chunk limits and NLI truncation at 512 tokens constrain visibility; they do not summarize an accumulating conversation or guarantee full-document coverage. Ordinary SFT/DPO evaluation omits documents, whereas RAG supplies the current record's document. The exposed embedding/topic-cache helpers are a separate, inactive alternative family, not evidence that active generation performs semantic memory retrieval (RT-RTE-data-generation, RT-RTE-response-generation, MEM-OBJ-topic-caches).
Scores control optimization, not factual warrant
The claimed distinction between context-conditioned consistency and context-free knowledge scoring is implemented through different inputs and numerical scoring. Its factual interpretation remains unvalidated. SelfCheckGPT estimates contradiction among answer texts; BSDetector adds entailment estimates and optional self-rating; semantic clustering uses two-direction noncontradiction, including neutral predictions, rather than the docstring's bidirectional-entailment criterion. Consistent false answers remain a compatible countercase to equating low contradiction with knowledge. These checks have different targets and cannot be combined into a system-wide factuality verdict (RT-CLM-knowledge-filtering, EPI-RTE-score-evidence, EPI-CLM-semantic-clustering).
After operator launch, individual admissions are automatic. Stored scores rank and exclude examples where configured thresholds apply. A positive unknown threshold selects from the question-filtered subset; a nonpositive threshold falls back to the full dataset, bypassing that subset. With reference-as-chosen enabled, extracted gold_answer supplies the single-preference pair and first multiple-preference pair. Later multiple-preference pairs instead use selected gold_answer_sample entries. The flag therefore does not guarantee reference identity for every admitted pair, and no branch guarantees reference truth (RT-RTE-model-training, EPI-RTE-preference-selection).
The trainer receives formatted prompt/chosen/rejected columns, not the complete document and score provenance used by the selector. Scores can remain in source files, but saved weights are not an inspectable rationale store. Rescoring replaces retained score fields; known-subset persistence and aggregate reporting reduce existing material. Established later accesses are request-driven file, model and cache loads. A later consumer of terminal reports is not established, so these routes do not justify an inventory-wide pull-only or no-push claim (MEM-OBJ-answer-scores, MEM-OBJ-known, MEM-OBJ-results).
Deliver a checkpoint, then a bounded comparison
Training completion saves a checkpoint without requiring a subsequent judge win. Direct inference loads a configured checkpoint path. TGI instead calls an independently launched endpoint; its shipped launcher defaults to base TinyLlama. A DPO-labeled response file therefore does not attest DPO weight consumption. The operator must align server weights, tokenizer and experiment paths (RT-CMP-causal-model, RT-RTE-response-generation).
The evaluator joins saved responses with baseline responses and test documents, asks GPT-4o for both answer orders, and parses win/lose/tie labels. Invalid judgments receive bounded retries; comparisons without two valid choices are omitted. Proportions cover counted results, not necessarily every requested question; zero counted results lacks a division guard. The exact-prompt cache bypasses new judgments but omits system-prompt and actual weight revision from its key. Printed hallucination zeros are disabled-branch placeholders, not measurements. The appended comparison report ends the selected workflow: it does not automatically veto, promote, retrain or roll back a checkpoint (RT-RTE-pairwise-evaluation, MEM-OBJ-judge-cache, workflow evaluation annotation).
For a bounded, operator-managed Wikipedia training experiment, the source establishes consequential selection, parameter updating and a later consumer. For an unchanged-wrapper fresh start, it does not establish delivery: the default scorer spelling is invalid, DPO generation requires prior questions, fresh test initialization indexes an empty list, and the testing wrapper expands unassigned local variables. Question-filtering flags also do not by themselves connect generation's _qf answer file to training's unsuffixed dataset path (RT-RTE-data-generation, memory integration issues, workflow account).
Independent learning and self-change conclusions
The positive trace-fed update pathway does not establish improved capacity for future action. At the builder boundary, over operator-mediated generation/training/inference invocations, the independently stated objective is more truthful, question-relevant Wikipedia responses. Evidence-based selection, parameter change and a later behavioral-authority path are wired; subsequent behavior causally depending on that update is unobserved. This supports dispositional improvement-directed self-change, not exercised or successful self-improvement (RT-RTE-model-training, RT-BAP-checkpoint-consumption).
Theory-builder conditions 1–4 are not established by these records: they identify no localized formulated theory for this assessment, no demonstrated decision dependence on such theory content, no stated content-directed criticism with blame and changed reliance, and no retained criticism shaping a next round. This is not an absence claim about opaque model processing. Separately, criticism of a consumed theory improving future capacity is unestablished. Numerical selection and parameter updates alone do not establish it. Reflection is likewise unestablished, not disproved; a score about answers does not alone demonstrate the required self-representation and both causal directions. The end-to-end workflow is operator-mediated, not autonomously owned by computation across its stages. These conclusions do not negate the implemented update mechanism (RT-RTE-model-training, epistemic bounded conclusion).
A provenance-linked run showing admitted examples, saved weight identity, actual inference consumption and changed subsequent behavior would resolve the missing exercise link. Independent factual references and a controlled baseline comparison would address improvement; independent variation of filtering or other components would be needed for their causal effects. Fixing launcher defaults, dataset-path continuity and endpoint identity controls would change the delivery assessment, but would not itself demonstrate benefit. Both independent verifications report no blockers or additional declared limits; their acceptance does not convert source wiring into observed operation (report verification, profile verification).
Limitations
| Limitation | Affected SRC IDs and record citations | Inspected boundary | Conclusion prevented | Evidence that would resolve it |
|---|---|---|---|---|
| No supplied execution or comparison artifacts | SRC-1; RT-RTE-model-training, RT-BAP-checkpoint-consumption | Shipped generation, training and later inference wiring | Exercised learning, update-dependent behavior, successful self-improvement or a measured causal gain | Linked data/training/inference traces, weight identities and controlled outcome comparisons |
| Fresh-start and launcher defects | SRC-1; RT-RTE-data-generation, RT-RTE-response-generation | Generation defaults, prerequisite questions/test initialization and testing wrapper | Unconditional unchanged-wrapper end-to-end delivery | Corrected launchers and a trace covering initialization and stage completion |
| Dataset continuity and initial provenance are conditional | SRC-1; MEM-OBJ-answer-content, memory integration issues | File-existence reuse, exact-instruction remapping and _qf producer/consumer paths |
Authenticated lineage, complete question preservation or flag-only dataset continuity | Content/version manifests, mapping coverage and resolved producer/consumer paths |
| Reference and score validity are not independently established | SRC-1; EPI-OBJ-gold-answer, EPI-RTE-score-evidence | Generated labels, answer-pair NLI, self-rating and noncontradiction clustering | Factual acceptance, calibrated knowledge measurement or entailment-guaranteed clusters | Independent references, calibration and checked input/output cases for each method |
| Selection guarantees are branch-local | SRC-1; RT-RTE-model-training, EPI-RTE-preference-selection | Threshold fallback, later multiple-preference pairs and alternate scorer field compatibility | Universal question-filter enforcement, all-pairs reference identity or successful DPO use of every scorer | Explicit branch invariants, compatible schemas and branch-specific traces |
| Actual model identities and external behavior remain unresolved | SRC-1; RT-CMP-causal-model, RT-RTE-response-generation, RT-CMP-judge-model | Mutable checkpoint paths, unpinned dependencies and separately selected TGI service | Attested checkpoint consumption, reproducibility or attribution of provider reliability | Immutable model/dependency identities, server attestation and request-linked artifacts |
| Context and cleanup do not preserve complete evidence | SRC-1; EPI-RTE-document-transform, EPI-RTE-answer-cleaning, MEM-OBJ-answer-scores | Document-tail deletion, chunk selection, lexical cleanup, NLI truncation and DPO formatting | Full source coverage, universal semantic preservation or rationale read-back from weights | Retained transformation lineage, semantic checks and an implemented later evidence consumer |
| Evaluation is selective and cache-relative | SRC-1; RT-RTE-pairwise-evaluation, MEM-OBJ-judge-cache | Two-order model rankings, failed-comparison omission, cached categories and placeholder zeros | Absolute factual accuracy, all-question win proportions or renewed validity after judge changes | Complete denominators/replies, independent correctness checks and revision-aware rejudging |
| Alternative and opaque memory parts remain incompletely classified | SRC-1; MEM-OBJ-topic-caches, MEM-OBJ-answer-content, MEM-OBJ-results | Exposed helper caches, extra generation payload fields, initialization and possible later report readers | Active semantic retrieval, complete form/lineage/authority/curation inventory, inventory-wide pull-only classification or a controlled trace-origin category | Concrete payload and provenance records, helper caller wiring and established later consumers/original trace structure |
| Formulated-theory and reflective properties are not established | SRC-1; RT-RTE-model-training, epistemic bounded conclusion | Inspected numerical selection/update chain; opaque model processing remains unresolved | Theory-builder conditions 1–4, learning through theory criticism or reflection | Identified theory content, content-directed criticism and retained consequences; representation-mediated causal evidence for reflection |