用引用量作为奖励信号训练 LLM 生成高影响力科研想法
Learning to Ideate for Scientific Impact
这帮人拿 10 万篇 CS 论文的引用量当奖励，训练 LLM 生成更可能被高引用的科研点子，还专门设计了防作弊的评估办法，思路挺有意思。
论文构建了基于 100K+ 篇计算机科学论文的数据集，为每篇论文按年份归一化的引用量打序数标签。研究训练了一个目标条件奖励模型，从研究目标与想法的配对预测引用影响力。通过监督微调加强化学习对齐想法生成器后，RL 调优模型生成想法的预估影响力持续高于基座模型和 SFT 基线。评估采用留出集、以历史想法为参照的协议，按参照想法的引用影响力加权判断以降低循环性。
Learning to Ideate for Scientific Impact
Scientific ideation is increasingly mediated by large language models, but current ideation systems are usually trained and evaluated on immediately judgeable proxies such as novelty, clarity, and feasibility. This leaves open whether delayed signals of scientific uptake can be used as feedback for steering models toward research directions with higher expected \emph{impact}. We study this question using citation-normalized impact as a noisy but scalable proxy for scholarly uptake. We construct a large-scale dataset from over 100K computer science papers by extracting goal-conditioned idea descriptions and assigning each paper an ordinal, year-normalized citation label. We then train a goal-conditioned reward model to predict citation-impact labels from research goal and idea pairs, and use this reward to align an idea generator through supervised fine-tuning followed by reinforcement learning. To reduce circularity, we evaluate generated ideas with a held-out, reference-grounded protocol that compares model outputs against historical ideas under the same research goal and weights judgments by the reference idea's citation-impact label. Experiments show that our RL-tuned model consistently produces ideas with higher estimated impact than both the base model and supervised fine-tuning baselines. Our findings position scientific impact as a practical, outcome-grounded feedback signal for aligning LLMs in open-ended scientific discovery.
