Choice2Seq 基准：模型选得对答案，却难以一步步构造出来
Correct Choices, Failed Sequences: Choice2Seq Benchmarks Correctness Retention Across Output Interfaces
论文给模型做了个体检：选对答案不代表能一步步做出来，Jev 选对率 97% 却只剩 33%。
arXiv 论文提出 Choice2Seq 诊断基准，用序列组合率（SCR）衡量模型在整题选择正确后，能否通过逐步局部构造复现正确答案。Jev 在算术和词汇检索上的选择准确率分别达 97.1% 和 99.5%，但串行构造下只保留 33.0% 和 2.1%。StartLux 和 Qwen 也存在明显落差，Qwen 即使直接逐字符输出仍无法消除差距。实验显示，把局部选项限制为候选支持的字符可改善构造，显式位置索引能提升复制保真度；对话状态评测还发现局部字段构造基本保留 Jev 的正确状态，却降低了 Qwen 的平均轮次准确率。
Correct Choices, Failed Sequences: Choice2Seq Benchmarks Correctness Retention Across Output Interfaces
A model may select the correct answer yet fail to construct it through a sequence of local decisions. We introduce Choice2Seq, a diagnostic benchmark that measures correctness retention across complete-answer selection and sequential construction in independent runs on the same problems and candidate sets. Sequence Composition Rate (SCR) measures the fraction of correctly selected items that are also answered correctly through local construction. Jev achieves 97.1% and 99.5% selection accuracy on arithmetic and lexical retrieval, yet retains only 33.0% and 2.1% of those correct choices under serial construction, respectively. Substantial gaps also emerge for StartLux and Qwen, including when Qwen emits characters directly. Restricting local options to candidate-supported characters improves construction, while explicit position indexing improves copying fidelity, even for incorrect selections. These interventions distinguish task correctness, plan fidelity, and output compliance. Matched dialogue-state evaluations reveal that local field construction largely preserves Jev's correct states but reduces Qwen's mean-turn accuracy, highlighting task- and interface-dependent differences. Sequence-level confidence and request-cost analyses further characterize completed outputs.