h t t p s : / / a r x i v . o r g / a b s / 2 6 0 8 . 1 4 7 9 1

Ingest: CEDAR-GRPO: Process-Aware Reinforcement Learning for General Abductive Reasoning in LLMs

Type: kb/sources/types/ingest-report.md

Classification

An arXiv v1 preprint that specifies a post-training method, controlled comparisons, held-out evaluations, and reproducibility details. Author: A nine-author academic team from Sharif University of Technology and the University of Tehran. The paper provides detailed prompts, reward definitions, dataset composition, hyperparameters, and a code-and-data link, but this ingest did not inspect the repository or independently reproduce the experiments.

Summary

CEDAR-GRPO post-trains four open-weight 4--8B models on 1,920 training examples from seven datasets grouped as hypothesis generation or hypothesis selection. Its GRPO reward equally combines task-specific final-answer correctness with two gpt-oss-120b judgments over the emitted reasoning trace: coverage of observation details and evidence-to-explanation directionality. Across 3,800 examples from 11 held-out tasks, the paper reports that every CEDAR-GRPO checkpoint beats its base and correctness-only GRPO counterparts, averaging gains of 7.4 and 2.7 percentage points respectively. On one backbone, a separate Gemini 3 Flash evaluation also finds higher average coverage, directionality, branchiness, backtracking, differential elimination, prior invocation, and uncertainty marking. Reward, curriculum, SFT, and generic-reasoning comparisons support the compound training design, but do not establish every component or the paper's fixed definition of abduction independently.

Quotes

  • Source extract (verbatim): Models generate structured outputs of the form ⟨think⟩ β ⟨/think⟩⟨answer⟩ α ⟨/answer⟩, where β is the reasoning trace and α is the final answer. Task correctness is computed from α, whereas the process rewards are computed from the user prompt and β.
  • Source location: Section 4.2, "Structured CoT Prompting"
  • Source extract (verbatim): We compare the original base model, the correctness-only GRPO checkpoint (denoted Cor- GRPO), and our main composite-reward check- point, CEDAR-GRPO.
  • Source location: Section 5.1, opening comparison; capture retains line-break hyphenation
  • Source extract (verbatim): No additional transformation is applied before GRPO. Cor-GRPO uses only rcor.
  • Source location: Appendix D.1, "Composite Objective"
  • Source extract (verbatim): and 2.7 points over Cor-GRPO
  • Source location: Introduction, reported average improvement over the correctness-only comparator
  • Source extract (verbatim): Baseline 0.69 1.22 0.79 0.59 0.87 33.1% 0.21
  • Source location: Table 3, DeepSeek-R1-Distill-Qwen-7B mean process metrics; columns are Backtracking, Branchiness, Differential Elimination, Prior, Uncertainty, Coverage, and Directionality
  • Source extract (verbatim): Cor-GRPO 0.93 1.16 0.97 0.53 0.92 39.1% 0.16

  • Source location: Table 3, DeepSeek-R1-Distill-Qwen-7B mean process metrics; same columns

  • Source extract (verbatim): CEDAR-GRPO, a process-aware framework that combines final-answer correctness with abductive rewards for evidence coverage and evidence-to-explanation directionality.

  • Source location: Abstract
  • Source extract (verbatim): Our experiments use four open-weight backbones:
  • Source location: Section 4.1, “Model Families”
  • Source extract (verbatim): We evaluate them on 11 unseen tasks
  • Source location: Abstract
  • Source extract (verbatim): to a total number of 2400 samples, consisting of 1920 training and 480 validation samples.
  • Source location: Appendix A, Table 5 caption
  • Source extract (verbatim): Each reasoning trace is passed to Gemini 3 Flash together with a structured system prompt specific to the metric and a short dataset-specific contextual framing.
  • Source location: Appendix E.1, “Evaluation Framework”

Connections Found

The source is a bounded empirical anchor for process structure and output structure as independent levers: correctness-only and CEDAR training retain the same <think>/<answer> contract, while trace-directed rewards change held-out final-answer performance. Reading that result rests on both the fixed-decomposition boundary and the rule that an experiment identifies only its observed contrast, because the taxonomy, prompts, judges, model update class, and benchmark interpretations remain supplied, while several ablations vary bundles or reweight retained rewards. As a process-evaluation source, it is also a useful counterpoint to intervention evidence that emitted reasoning can be unfaithful and to FALSIFYBENCH's externally enacted hypothesis tests: CEDAR measures useful verbal trace properties, not the causal faithfulness of those traces or agent-chosen operations in an environment.

Extractable Value

  1. Behavioral efficacy and trace faithfulness are separate claims -- CEDAR shows that optimizing text-level process judgments can improve held-out answers, but it does not show that the rewarded text faithfully reports the computation producing those answers. That distinction is a higher-reach synthesis not yet stated directly in the KB. [deep-dive]

  2. Process-directed optimization can affect outcomes beyond a fixed output contract -- CEDAR and correctness-only GRPO use the same structured response shape, yet the composite reward gains 2.7 points on average and the two partial reward bundles are weaker than the full bundle on the two ablation backbones. This extends the KB's process/output distinction from inference-time templates to weight updates. [quick-win]

  3. Final-answer improvement can coexist with weaker measured reasoning traits -- on DeepSeek-R1-Distill-Qwen-7B, correctness-only GRPO improves task accuracy while average directionality falls from 0.21 to 0.16, branchiness from 1.22 to 1.16, and prior invocation from 0.59 to 0.53. This is a concrete warning against treating outcome accuracy as a sufficient process-health signal. [quick-win]

  4. The paper supplies reusable process-oracle designs -- its appendices give zero-shot judge prompts and operational definitions for coverage, directionality, branchiness, backtracking, differential elimination, prior invocation, and uncertainty marking. These are useful candidate measurements for experiments, provided they are calibrated rather than imported as validated constructs. [experiment]

  5. The ablation grid is a compact contrast-accounting case -- removing a reward also changes the relative weights of retained rewards; stage-only arms change curriculum bundles; and RL versus SFT changes both optimization and synthetic-rationale supervision. The table is useful for teaching how component language can exceed treatment grain even in an unusually thorough ablation section. [just-a-reference]

Limitations (our opinion)

The effective update space is narrower than the headline capability claim. Behaviour can condition on the fixed task prompt and model state, and the training oracle sees the prompt, emitted trace, final answer, and task-specific verifier result. The model can compose token sequences within a structured trace/answer interface, including code for the executable tasks, but cannot choose new evidence-gathering operations or revise the task decomposition. The learnable mappings are rank-64 LoRA adaptations of four NF4-quantized 4--8B transformers. Fixed outside that space are the generation/selection taxonomy, dataset mixture, prompt templates, trace representation, equal-weight reward, judge rubrics, verifier definitions, model scale, and evaluation framing. Improvement inside this configuration does not validate those fixed choices or the decomposition as a whole.

The reported results use fixed random seeds and point estimates, with no repeated-training uncertainty or significance analysis. Some of the claimed wins on individual 250- or 400-example tasks are only one or a few correct answers, so “every model on every task” describes these checkpoints and samples rather than established run-to-run robustness. Dataset holdout is real, but not every evaluation is a strong distribution shift: Balanced COPA cause examples appear in training and its effect split in evaluation, most outcomes are closed-form, and several “domain-abductive” tasks count as abduction by an interpretive choice. The paper itself acknowledges the absence of open-ended interactive tests, models above 8B, larger training sets, and human process evaluation.

The process evidence remains soft. Training and evaluation use different judge models, which reduces exact self-scoring circularity, but both apply author-defined constructs to emitted text without a human-labelled calibration set. Five reported process metrics are raw occurrence counts per trace and are not normalized for trace length, leaving verbosity as a simpler explanation for part of their increase. More fundamentally, a fluent trace can omit or rationalize the computation that caused an answer. The directionality rubric also treats hypothesis-to-evidence checking as a failure, although generating a candidate and testing its predictions can be a legitimate part of abductive comparison; the reward may therefore teach preferred rhetorical order rather than a uniquely valid reasoning process. The advertised code and data were not inspected or executed here, so all implementation and outcome claims remain paper-only.

Update Process structure and output structure are independent levers with one bounded CEDAR evidence paragraph: record the fixed <think>/<answer> contract, the composite-versus-correctness-only held-out delta, and the accuracy/process divergence, while stating explicitly that the experiment supports behavioral efficacy of trace-directed rewards rather than trace faithfulness or the validity of its directionality construct.