PersonaHate:基于人设的数据合成管线,生成 791K 仇恨言论数据集
PersonaHate: A Scalable Persona-Based Data Synthesis Pipeline for Hate Speech Analysis
一篇很实用的数据合成论文:8 个 LLM 按人设生成了 79 万条仇恨言论样本。用它的 6.7 万条数据微调后,Llama-3.1-8B 的 F1 比 Meta 自家 LlamaGuard-3 还高。做内容审核或安全数据集的团队直接照着抄管线就行。
arXiv 论文 PersonaHate 提出一条可扩展的人设驱动数据合成管线,用于构建覆盖面更全的仇恨言论数据集。管线从两个真实来源抽取 457K 人设池,以此条件化 8 个 LLM 生成针对 34 个身份群体的仇恨内容,并经多裁判模型标注获得可靠标签,最终得到 791K 样本语料,再从中整理出 67K 的群体均衡训练集。
机制上,PersonaHate 用人设作为生成条件的锚点,避免合成数据集中于少数目标群体;模块化设计允许接入新的身份群体、生成器和新出现的仇恨对象。基于该数据集,论文比较了从零训练和继续微调两种策略:一个 184M 参数的 DeBERTa-v3 模型达到 77.5% 平均 F1 和 79.4% 平均准确率,在 HateBenchSet 和 MHS 两个外部基准上超过 OpenAI 的 Moderation API。
对开源生态的更直接信号是,微调 Llama-3.1-8B 将其平均 F1 从 70.7% 提升到 78.4%,提高 7.7 个百分点,并在 4 个基准中的 3 个上超过 Meta 官方同架构的 LlamaGuard-3。这说明中小模型配合高质量合成数据,在内容审核这类任务上可以逼近甚至超过专用的安全模型,对自建审核管线的内容平台有参考价值。
PersonaHate: A Scalable Persona-Based Data Synthesis Pipeline for Hate Speech Analysis
Hate speech remains a persistent and evolving threat on online platforms. The widespread use of large language models (LLMs) further intensifies this problem by making hateful content easier and cheaper to generate at scale. Although stronger hate speech detectors continue to be proposed, their performance remains uneven across target identity groups and different forms of hateful expression, partly because existing training data provides only limited coverage of these cases. Thus, we introduce PersonaHate, a scalable persona-driven data synthesis pipeline for constructing diverse hate speech datasets. PersonaHate begins with a large persona pool of 457K personas drawn from two real-world sources. Conditioned on selected personas from this pool, it prompts eight LLMs to generate hate speech across 34 identity groups and performs multi-judge annotation to obtain reliable labels. The resulting corpus contains 791K samples, from which we curate a 67K group-balanced training set. Our modular design also supports extension to new identity groups, generators, and emerging hate targets. Built upon this dataset, we evaluate two detector training strategies: training from scratch and further fine-tuning. Results show that all trained models achieve competitive performance across four widely used external benchmarks. Notably, a 184M-parameter DeBERTa-v3 model trained on PersonaHate reaches 77.5% average F1 and 79.4% average accuracy, outperforming OpenAI's Moderation API on HateBenchSet and MHS. Furthermore, fine-tuning Llama-3.1-8B raises its average F1 from 70.7% to 78.4%, a gain of 7.7 percentage points, surpassing LlamaGuard-3, Meta's official safety model based on the same Llama-3.1-8B architecture, on three of four benchmarks. These results show that PersonaHate can substantially improve the robustness and cross-dataset generalization of hate speech detectors.
本条情报归属 AITOP「论文」核心追踪栏目。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:PersonaHate、DeBERTa-v3、Llama-3.1-8B、LlamaGuard-3,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始信息发布自一手权威信源「arXiv: OpenAI」(Tier-1 一手发布),具备高确定性参考价值。