论文

arXiv 论文实证审计:LLM 简历筛选系统的部署就绪度

Can AI Judge You? An Empirical Audit of Deployment Readiness in LLM-assisted Hiring Systems

精选理由

四位研究者拿210份简历实测了4个LLM做简历筛选的靠谱程度,发现打分比通过/不通过更准,模型的决定和理由还经常对不上。想做AI招聘的企业和HR工具开发者,这篇给出了能直接抄的审计清单。

一篇 arXiv 论文构建了将候选人证据、评估、权限控制和评估记录相互分离的简历筛选系统架构,并对两个维度的部署就绪度做实证检验。实验在 4 个 LLM 上进行,使用 210 份简历、14 个岗位角色和 6 种决策接口。结果显示,标量分数比简单的通过/不通过判断更能区分资质变化,而比较型判断对呈现顺序敏感。

正文

合规维度上,显式权限规则和将简历内容与指令分离能降低未授权请求的通过率,但在不同 LLM 和使用场景下效果不一。论文还发现 LLM 评估器给出的决定与其提供的理由经常不一致。

论文将评估对象从单个模型扩展到完整配置的简历评估系统,为企业在部署前提供审计框架。对 HR 科技厂商和采用 LLM 做招聘筛选的企业来说,接口设计和权限隔离的细节直接影响实际筛选质量。

原文 · arXiv cs.AI

Can AI Judge You? An Empirical Audit of Deployment Readiness in LLM-assisted Hiring Systems

Delegating resume screening to large language models (LLMs) requires organizations to establish whether they can distinguish candidates by relevant qualifications while allowing only authorized users to change screening rules or approve exceptions. To this end, we develop a system architecture that separates candidate evidence, evaluation, permission controls, and assessment record, and empirically examine two complementary dimensions of its deployment readiness. First, we assess its operational readiness on four LLMs using a controlled experiment on 210 resumes across 14 job roles and six decision interfaces. Second, we assess its compliance readiness by testing how it handles unauthorized user requests and instructions embedded in resumes. We find that operational performance depends on the choice of evaluation interface: scalar scores generally distinguish qualification changes better than simple pass/fail judgments, while comparative judgments are sensitive to presentation order. Explicit permission rules and separation of resume content from instructions reduce unauthorized compliance, to a varying degree of effectiveness across LLMs and usage contexts. Finally, we find that LLM evaluators often diverge between their decisions and offered justifications. Going beyond simply evaluating LLMs for hiring tasks, our framework supports an audit of a fully configured resume evaluation system to enable its deployment decision.

产业坐标与信源核验

本条情报归属 AITOP「论文」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:LLM、AI招聘、简历筛选、审计,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。

信源核验说明:本文原始线索采编自「arXiv cs.AI」(@Spandan Roy, Subhabrata Majumdar, Soudeep Deb),经 AITOP 多维评分管线结构化提炼。