技巧83°

图像模型后训练奖励设计方法

精选理由

Arena团队发布了图像模型后训练新方法,通过复合奖励显著提升FLUX和Ideogram性能,解决奖励攻击问题。

研究提出复合奖励机制,结合Bradley-Terry偏好模型、忠实度奖励、约束奖励和反奖励攻击标准。该方法使FLUX.

正文

2-dev在T2I排行榜上提升69分达1202分,Ideogram 4提升20分达1224分超越所有开源模型。Gemini 3.

5 Flash评估显示,添加忠实度和约束奖励后胜率提升至64.2%,集成策略后达66.

0%。

原文 · lmarena.ai

How to design rewards for post-training frontier image models? Our research suggests human preference reward is necessary, but insufficient: A preference model may still reward outputs that look appealing but miss details, introduce unrequested content, or exhibit other forms of reward-hacking. We therefore optimize towards a composite reward: - Bradley-Terry reward model trained on ~5.6M pairwise human votes - Faithfulness reward from auto-generated prompt checklists evaluated by a vision-language model - Constraint reward covering explicit and implicit user intent - Anti-reward-hacking rubric rewards targeting failures such as garbled text and photorealism drift This post-training recipe improves two already-strong open image models: - Post-trained FLUX.2-dev gains 69 Elo points on our live T2I leaderboard, scoring 1202 - Post-trained Ideogram 4 gains 20 Elo points reaching a score of 1224 and surpassing all publicly listed open models (as of Sep 04, 2026). Offline ablations with Gemini 3.5 Flash as the judge, show that these reward components are complementary: win rate against the base model increases as we add faithfulness and then constraint rewards on top of preference-only training, reaching 64.2%. Finally, we ensemble policies trained with and without the anti-reward-hacking objective directly in weight space, further increasing win rate to 66.0%.📊 11 ⚡

产业坐标与信源核验

本条情报归属 AITOP「技巧」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:FLUX、Ideogram、Arena、图像生成,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「lmarena.ai」(@lmarena_ai),经 AITOP 多维评分管线结构化提炼。