Authors introduce Step Rejection Fine-Tuning for trajectory distillation

Per the arXiv submission "Step Rejection Fine-Tuning: A Practical Distillation Recipe" (submitted May 11, 2026), the authors introduce Step Rejection Fine-Tuning (SRFT) as a modification of standard Rejection Fine-Tuning (RFT) for training large language models on multi-step agent trajectories. The paper reports that RFT, which discards trajectories that fail to resolve, yields a 2.4% improvement in resolution on SWE-bench Verified by excluding unresolved runs, while SRFT improves resolution by 3.7% and achieves a total resolution rate of 32.2%, according to the paper.

Per the paper, SRFT uses a separate critic LLM to assess correctness at the granularity of individual steps within a trajectory. During training the method retains the full trajectory in the model's context window but masks the loss for steps flagged as erroneous, so the model does not learn to reproduce those specific mistakes. The authors evaluate on SWE-bench Verified, a software-engineering-oriented benchmark, and report the numerical improvements cited above.

Converting unresolved trajectories into partially supervised training examples by masking loss at erroneous steps echoes selective-loss masking and targeted distillation techniques used elsewhere in model compression and imitation learning. For practitioners, the main practical benefit is more efficient use of generated trajectories: instead of discarding long, partially-correct episodes, selective masking preserves corrective signal while preventing error replication.

Editorial analysis: The paper addresses a practical bottleneck for improving agentic LLM performance on hard, long-horizon tasks where full success is rare. By increasing usable training signal from failed runs, SRFT targets sample-efficiency and robustness of multi-step generation, which are active concerns for research on code generation, theorem proving, and agent frameworks.

Editorial analysis: Replicability on other benchmarks and sensitivity to the critic's accuracy are the natural next checks. Observers should look for released code, critic model configurations, and ablations showing how masking thresholds and critic errors affect downstream generalization.