IDM 与 CMM 蒸馏方法提升并行生成式推荐效果
Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation
推荐系统方向的朋友可以看看这篇：两个蒸馏目标函数让并行语义 ID 推荐超过 SOTA，IDM 提 Recall@10 超 15%，思路挺巧。
生成式推荐用语义 ID（semantic IDs）把物品表示成离散码序列，但现有模型只用硬标签训练，缺少备选物品的分布信息。论文提出两种知识蒸馏目标：IDM 把学生并行码预测重建为物品分布并与教师对齐，CMM 把教师物品概率投影到各码位后匹配码边际。在基准数据集上，IDM 平均提升 Recall@10 15.39%，CMM 平均提升 NDCG@10 8.28%，均超过现有 SOTA 方法。
Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation
Generative recommendation with semantic IDs represents each item as a sequence of discrete codes derived from its semantic representation and predicts these codes to generate recommendations. Existing semantic-ID recommenders are typically trained with hard labels for the observed next item. Although such supervision identifies the target, it provides no explicit distributional information about alternative items conditioned on the same user history. Sequential recommenders can supply this item-level predictive knowledge, but transferring it to semantic-ID generation is challenging because the teacher and student operate in item and code spaces, respectively. We propose two complementary knowledge-distillation objectives for parallel semantic-ID recommenders, which predict all code positions simultaneously without autoregressive prefix conditioning. Item Distribution Matching (IDM) reconstructs a catalogue-level item distribution from the student's parallel code predictions and aligns it with the teacher's item distribution. Code Marginal Matching (CMM) projects the teacher's item probabilities onto each code position using fixed item-to-code assignments, then matches the resulting code marginals to the corresponding student heads. Both objectives transfer the teacher's soft distributional knowledge while preserving the student's parallel prediction structure. Experiments on benchmark datasets show that both methods consistently outperform existing state-of-the-art approaches. On average, IDM improves Recall@10 by 15.39%, while CMM improves NDCG@10 by 8.28%. These results demonstrate that distillation in either reconstructed item space or code-marginal space can substantially strengthen hard-label supervision for parallel generative recommendation.
