Weaviate 上线 Diversity Ranking:一个参数解决检索结果同质化
做 RAG 或搜索的人应该都有过返回五条几乎一样的结果的体验,Weaviate 把 MMR 多样性重排做成了一个 diversity_weight 参数,省掉自己写 rerank 流水线。和自己手搓方案比配置成本低很多,值得在你的检索链路里试一下 0.5 左右的取值。
Weaviate 推出 Diversity Ranking 功能,针对向量搜索、BM25 和 cross-encoder 都存在的同一问题:每篇文档只按与 query 的相关性打分,返回的 top-k 结果常常高度重复,比如问"史上最伟大运动员"会得到五名篮球运动员。
其实现基于经典的 Maximal Marginal Relevance(MMR)算法,在每一步选择时同时权衡候选结果与 query 的相关性,以及与已选结果的相异度,让结果列表覆盖整个主题而非单一角落。使用上只需设置一个参数:Weaviate 的 search 层支持 diversity_weight,取值 0.
0 到 1.0,越接近 0 越偏重相关性,越接近 1 越偏重多样性;
qa.search 接口可直接传入 diversity_weight=0.
5;Query Agent 中则在向量或混合查询上加 diversity_selection=Diversity.
mmr(..
.),其中 balance 参数方向相反(0.
0 偏多样性、1.0 偏相关性)。
此前开发者要自己实现多样性重排,需要超额取回大候选池、做候选间相似度检测、编写并调优独立的 reranking 阶段、长期维护这套流水线代码,现在这些步骤被收敛为一个参数配置。
Ever search your own data and get five results that are basically the same? Ask for "the greatest athlete of all time" and you might get five basketball players. Each result is relevant, but together they give you a very narrow answer. That's what 𝗗𝗶𝘃𝗲𝗿𝘀𝗶𝘁𝘆 𝗥𝗮𝗻𝗸𝗶𝗻𝗴 fixes. Vector search, BM25, and cross-encoders all score each document against the query and hand you the top-k. Nothing in that process asks "have I already returned something like this?" Diversity Ranking uses a classic idea, Maximal Marginal Relevance (MMR): at each selection step, pick the next result by balancing its relevance to the query against how dissimilar it is to what you've already picked. Your result list covers the topic instead of repeating one corner of it. Without it, you'd have to: 1. Overfetch a big candidate pool 2. Detect similarity or redundancy between candidates 3. Write your own reranking stage 4. Tune it 5. Maintain that pipeline code forever With it, you just need to set 𝗼𝗻𝗲 𝗽𝗮𝗿𝗮𝗺𝗲𝘁𝗲𝗿: diversity_weight The parameter runs from 0.0 to 1.0: values closer to 0 prioritize relevance, while values closer to 1 give more weight to div qa.search ts. https://t.co/WBFGwgX70P("the greatest athlete of all time", limit=5, diversity_weight=0.5) In the Weaviate , add `diversity_selection=Diversity.mmr(...)` to a vector or hybrid query. `balance` works in the opposite direction: 0.0 favors diversity, 0.5 balances both, and 1.0 favors relevance. Read mor docs.weaviate.io/query-agent/gu… ent: https: docs.weaviate.io/weaviate/searc… ase: https://t.co/FDEsDrn2Lm
本条情报归属 AITOP「产品」核心栏目,重点聚焦北美前沿实验室与硅谷科技巨头。在跨区域产业观察坐标中,该动态直接关联「AI 开发者工具链、Coding Agent 与工程化实践」「中美大模型技术代差与商业化对照」研究主线,反映出当前技术演进与商业落地的最新风向。涉及核心实体与技术议题包括:Weaviate、RAG、MCP/工具、检索,读者可通过下方标签进入对应专题页查阅历史脉络与关联对比。
信源核验说明:本文原始线索采编自「Weaviate」(@weaviate_io),经 AITOP 多维评分管线结构化提炼。截至目前,全网已有 3 家独立信源对该事件进行交叉印证与跟进报道(含 Databricks、shao__meng 等),属于近期高关注度行业信号。