SelectTTRL：用表征引导的数据选择降低测试时强化学习成本
SelectTTRL: Representation-Guided Data Selection for Efficient Test-Time Reinforcement Learning
TTRL 要对所有测试题都跑昂贵的采样和训练，这篇论文用 COS 指标提前筛掉不值得训练的题，算力省了性能还不掉，代码开源了。
SelectTTRL 是一种针对 Test-Time Reinforcement Learning（TTRL）的数据选择方法。论文发现模型内部响应表征与问题可解性之间存在稳定关系，据此定义 COS 指标——响应表征与基于独立问答数据集构建的"正确性方向"的余弦相似度。该指标在多个模型、数据集和实验设置中都与问题可解性相关，作者还给出了理论分析。SelectTTRL 用 COS 对测试问题排序，只对选定的百分位区间执行 TTRL，跳过其余问题的多响应采样和强化学习更新，在多个推理基准上降低计算量的同时保持或提升了标准 TTRL 的性能。代码已在 GitHub 开源。
SelectTTRL: Representation-Guided Data Selection for Efficient Test-Time Reinforcement Learning
Test-Time Reinforcement Learning (TTRL) enables large language models to adapt directly on unlabeled test problems by constructing pseudo-labels from multiple sampled responses. However, existing TTRL methods typically apply the same expensive sampling and optimization procedure to all test problems, despite substantial differences in their training utility. In this paper, we study whether such problems can be identified before costly test-time training, without access to ground-truth answers. We uncover a stable relationship between internal response representations and problem-level empirical accuracy. Specifically, we define COS as the cosine similarity between a model's response representation and a correctness-related direction constructed from an independent question-answering dataset. COS consistently correlates with problem solvability across different models, datasets, and experimental settings, and we further provide a theoretical analysis explaining this relationship. Building on this finding, we propose SelectTTRL, a representation-guided data selection method that ranks test problems using COS and applies TTRL only to a selected percentile interval. SelectTTRL leaves the standard TTRL pipeline unchanged for the selected problems, while avoiding unnecessary multi-response sampling and reinforcement learning updates on the remaining ones. Experiments across multiple reasoning benchmarks demonstrate that SelectTTRL substantially reduces test-time computation while maintaining or improving the performance of standard TTRL. Code is available at https://github.com/heyiliyi/SelectTTRL-public.
