研究：类型安全决策模型仍会受选项名称语义干扰
Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It
做结构化输出的人该看看：选项叫 0/1 还是 no/yes 能让 AUC 从 .94 掉到 .23，类型校验根本拦不住这种错。
arXiv 论文研究了 Jev 及两个同类型开源模型在结构化决策任务中的表现：模型输出保证符合 schema，类型错误率为 0%。但将 1200 个工作流决策中的两个选项名从 0/1 改为 no/yes 后，每百个答案中有 70.4 个发生变化，AUC 从 .94 跌到 .23，且该效应在全部 4 个谓词上比中性对照组大至少 7.4 倍。换成随机字符串作为选项名则可消除该效应且不损失准确率，说明失败源于选项名称的语义极性而非重命名操作本身。不同读出几何也有差异：对整个选项 span 做均值池化的第二个模型家族翻转频率低 4.1 倍。
Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It
Typed decision models are built for settings where model outputs are consumed directly by software. Instead of generating free-form text, they return a decision over a predefined set of options. By construction, every output conforms to the required schema. Yet this guarantee does not tell us whether the model interprets the options as intended. We study Jev and two Jev-like models with open weights by changing how option names are assigned to rubrics. Each option consists of an option name and a textual rubric that defines what the option means. We change only which option name is assigned to each rubric; the question, state, rubric wording, and set of option names remain exactly the same. On 1200 workflow decisions with task-specific rubrics, renaming the two options from 0/1 to no/yes changes 70.4 more answers per hundred (95% CI: [67.6, 73.1]) and shifts AUC from .94 to .23, revealing a systematic reversal in the decision ranking rather than simple uncertainty. The same operation has little effect with neutral option names. This pattern holds across all 4 predicates, where the effect is at least 7.4x larger than under the neutral control, and becomes stronger as the number of options increases. The effect also depends on the read-out geometry: a second model family that mean-pools over the full option span flips 4.1x less often. The hosted model exhibits the same behavior: the swap changes AUC from .8146 to .5806 and produces 24x as many answer flips as its test-retest floor. In contrast, replacing the option names with random character strings returns all model families to the neutral-control regime without reducing accuracy. The failure therefore depends on the semantic polarity of the option names rather than on the renaming operation itself. Across all conditions, the type-error rate remains 0%, even when decision accuracy degrades substantially.
