十款大模型幻觉与事实可靠性横评：Perplexity 与 DeepSeek V3 准确率最高
Fluent but Fallible: Benchmarking Hallucination and Factual Reliability in Large Language Models
一篇把 ChatGPT、Gemini、DeepSeek V3 等 10 个模型拉到同一起跑线测幻觉的论文，结论很反直觉：答得全的不一定准，Falcon 零幻觉但漏掉大半答案。
这项研究用 70 道学术提示词在 Basic、Explanation、Advanced、Tricky 四个难度层级上评测了 10 款大模型，包括 ChatGPT、Gemini、Claude 3.5 Sonnet、DeepSeek V3 等。每条回答由 3 名评审独立按 5 项指标打分，涵盖 Accuracy、Precision、Recall、Response Deviation Index 和 Hallucination Index。结果显示 Perplexity 和 DeepSeek V3 整体准确率最高，Gemini 在五项指标上表现均衡且 recall 强，LLaMA 3 因生成保守而幻觉最少。Falcon 达到零幻觉、precision 满分，但 recall 最低、响应偏差最高。Advanced 级提示词对所有模型都最难，Tricky 类因只有对错二元判断反而可靠。
Fluent but Fallible: Benchmarking Hallucination and Factual Reliability in Large Language Models
Large Language Models (LLMs) have significantly advanced natural language processing and artificial intelligence applications through their capabilities in reasoning, text generation and knowledge retrieval. However, LLMs are still vulnerable to two important challenges: hallucination, generating factually incorrect or fabricated content, and response deviation, meaning systematic deviations from ground-truth responses. These limitations raise serious concerns regarding reliability and trustworthiness in academic and research environments. This study offers a thorough comparative analysis using a variety of metrics of ten advanced LLMs ChatGPT, Gemini, Claude~3.5~Sonnet, LLaMA~3, Mistral, Grok, Perplexity, DeepSeek~V3, Falcon, and Gemma based on a benchmark consisting of 70 carefully designed academic prompts across four complexity levels: Basic, Explanation, Advanced, and Tricky, evaluated in February 2025 on specific model versions. Each response was independently evaluated by three reviewers using five metrics: Accuracy, Precision, Recall, Response Deviation Index, and Hallucination Index. Perplexity and DeepSeek~V3 achieved the highest overall accuracy, while Gemini demonstrated strong recall and a well-balanced performance profile across all five metrics. LLaMA~3 showed strong resistance against hallucinations through a conservative method of generation. Advanced-level prompts proved most challenging across all models, while Tricky prompts were handled more reliably owing to their binary true/false structure. Falcon achieved perfect precision and zero hallucinations, but at the cost of the lowest recall and the highest response deviation. The results show a fundamental trade-off between response completeness and factual reliability, intensified by prompt complexity.
