论文

arXiv 研究:分析 200 万条 LLM 引用数据,找出页面被引用的关键因素

What Drives Citations in Production Large Language Models? An Observational Multi-Method Study of Two Million AI Citations Across Ten Thousand Web Pages

精选理由

搞 SEO 或 AEO 的朋友该看看:分析了 200 万条 ChatGPT、Claude 等引用数据,发现堆 FAQ、加结构化数据没啥用,内容和用户提问匹配度才是关键。

一项研究分析约 200 万条 LLM 引用数据,覆盖 ChatGPT、Claude、Google AI、Gemini 四个引擎六个月的表现,并结合 10,000 个 B2B SaaS 网页。研究用九种统计方法检验 60 多个页面特征,其中页面内容与提示词语料的 Jaccard 重叠度是最强预测因子(beta = +0.37)。常见的 AEO 清单做法(FAQ 区块、结构化数据、Core Web Vitals)在加入域固定效应后效果归零或反转,出现辛普森悖论。域级 AI 权威度的 SHAP 值比最强的非对齐页面特征高约六倍。

原文 · arXiv cs.AI

What Drives Citations in Production Large Language Models? An Observational Multi-Method Study of Two Million AI Citations Across Ten Thousand Web Pages

Production large language models retrieve and cite web pages alongside generated answers, yet the page-level features that predict citation frequency remain poorly characterised. We present an observational study of approximately 2 million LLM citations from four commercial engines (ChatGPT, Claude, Google AI, Gemini) over six months, joined to 10,000 crawled pages from nineteen B2B SaaS workspaces. Sixty-plus features are tested using a nine-method consensus framework combining mixed-effects regression with domain fixed effects, FDR correction, stability-selection Lasso, double machine learning, generalised additive models, and temporal hold-out replication. Four findings survive all checks. First, prompt-content alignment (Jaccard overlap between page tokens and the full workspace prompt corpus, including non-citing prompts) is the dominant page-level predictor (beta = +0.37, 95% CI [+0.33, +0.41], q ~ 10^-73). Second, the standard AEO checklist (FAQ blocks, structured data, Core Web Vitals) shows positive effects in pooled data that reverse or collapse to zero once domain fixed effects are applied: Simpson's paradox with practical consequences for the AEO literature. Third, domain-level AI authority exceeds the strongest non-alignment page-level feature by a factor of six in mean absolute SHAP value. We release the analytic pipeline as a methodological contribution.