Apple 研究：LLM 做机器翻译质量标注，人类和模型都吃力
Large Language Models for Machine Translation Quality Annotation: Humans and Models Are Both Challenged
Apple 拿 LLM 和人工标注对比翻译质量评估，发现模型和人都没那么可靠，做翻译评估的可以看看。
Apple 发表论文，评估 LLM 在机器翻译质量评估中的表现，涵盖 MQM（多维质量指标）和 ESA（错误片段标注）两种主流评估方案。研究通过对比模型与人类标注者的一致性，覆盖多语言对、多领域和不同标注粒度。结果显示 LLM 想替代人工评估还不稳定，人类标注者之间的一致性同样面临挑战。
Large Language Models for Machine Translation Quality Annotation: Humans and Models Are Both Challenged
Large Language Models (LLMs) are considered to be a more efficient and cost-effective alternative to human judgment for Machine Translation (MT) evaluation. With MT evaluation spanning a large number of language pairs, domains and levels of annotation granularity, LLMs must be thoroughly evaluated across these dimensions before being reliably used as alternatives to human evaluation. In this paper, we evaluate the performance of LLMs for two prominent MT quality evaluation schemes: Multidimensional Quality Metrics (MQM) and Error Span Annotation (ESA) by comparing their agreement with human…
