arXiv 研究：概率相关性评分在 BEIR SciFact 上优于纯稠密检索
Probabilistic Relevance Decisions for Semantic Retrieval
做 RAG 检索的可以看看：同一套 BEIR SciFact 查询下，加一层概率相关性重排，nDCG 从 0.64 提到 0.78，还给了成本和延迟的实测数据。
该研究在 BEIR SciFact 基准的 300 条官方测试查询上，比较纯稠密检索 BGE-M3 与两种概率相关性决策评分（TypeSafe Jev Noul 和 OpenAI Decisions Predicate）。在全库检索加重排的条件下，深度 50 时的 nDCG@10 分别为 BGE-M3 0.6413、OpenAI 0.7545、Jev 0.7805，Jev 的 Recall@5 和 nDCG@10 最高。把候选深度从 50 扩到 100 能提高已知正例召回，但配对自助法显示最终效果变化不显著。API 评分成本约为原来的两倍，缓存可降低增量开销；研究结论是相关性决策可作为稠密检索的补充，不构成普遍优越性证明。
Probabilistic Relevance Decisions for Semantic Retrieval
Dense similarity supplies reusable corpus representations, while probabilistic relevance decisions offer query-dependent document scoring. We compare dense-only BGE-M3 with TypeSafe Jev Noul and OpenAI Decisions Predicate scores on the official BEIR SciFact test benchmark. The study evaluates 300 official test queries using identical positive-enriched 100-document pools and full-corpus dense retrieval followed by reranking at candidate depths of 50 and 100. Unlike the controlled pools, neither dense prefix receives positive injection. Jev has the highest observed Recall@5 and nDCG@10 under the evaluated conditions. At depth 50, full-corpus nDCG@10 is 0.6413 for BGE-M3, 0.7545 with OpenAI and 0.7805 with Jev. Extending depth from 50 to 100 increases known-positive candidate recall but produces small, uncertain final-effectiveness changes. All four paired-bootstrap depth intervals include zero. Unique-condition API scoring cost approximately doubles, while compatible cached decisions reduce incremental expenditure. OpenAI returns many exact-zero scores, and secondary exact-tie randomization reveals ranking sensitivity without replacing deterministic primary results. Top-100 latency mixes cached and new execution. Historical hybrid latency is a saved-component sum under different execution environments and timer boundaries. These exploratory, unadjusted analyses support relevance decisions as a complement to dense retrieval within this protocol, without establishing general superiority, probability calibration or equivalence between depths.
