Reasoning From Scratch 第 8 讲：用蒸馏替代强化学习训练推理模型
Raschka 推理模型系列的完结篇，手把手教你用 DeepSeek R1 的数据把 Qwen3-0.6B 蒸馏出推理能力，还能跑通 MATH-500 评估。
Sebastian Raschka 的 Reasoning From Scratch 系列更新第 8 讲，也是收官篇。本讲以 DeepSeek R1 MATH 数据集为例，讲解软蒸馏与硬蒸馏、on-policy 与 off-policy 蒸馏的区别。实践部分基于 Qwen3-0.6B 基座模型，将序列限制在 2048 token 内完成蒸馏训练。最后在 MATH-500 基准上评估各 checkpoint，并与教师模型结果对比。
Reasoning From Scratch round 8, the finale! This last lecture in the series covers distillation as an efficient alternative to the previous reinforcement learning training runs.
(Planning to take a brief break, but I will be back!)
00:00 Introduction and series recap 01:07 Why distill a reasoning model? 02:51 Soft vs. hard distillation 11:06 On-policy vs. off-policy distillation 18:09 Preparing data for reasoning distillation 22:35 Loading the DeepSeek R1 MATH dataset 28:04 Inspecting reasoning traces and final answers 31:27 Tokenization and chat templates 35:58 Pre-tokenizing the training dataset 39:01 Sequence lengths and the 2,048-token limit 42:38 Creating training and validation splits 45:32 Loading the Qwen3-0.6B base model 46:24 Computing answer-token log probabilities 54:10 Cross-entropy loss for distillation 57:25 Training and validation loss helpers 1:02:03 Implementing the distillation training loop 1:07:26 Running the standalone training script 1:09:41 Plotting training and validation losses 1:11:31 Evaluating checkpoints on MATH-500 1:14:09 Comparing teachers and distillation results 1:16:34 Future directions for reasoning models 1:19:41 Bonus exercises and next steps
