行业

Artificial Analysis 改用闭源基准测试,发现开源模型差距扩大

精选理由

Artificial Analysis 把公开基准换成闭源测试,结果开源模型落后更多了,关心开源和闭源差距的可以看看。

Artificial Analysis 将评估方式从公开基准 GDPval-AA 转向闭源基准,以减少数据污染带来的分数虚高。闭源测试结果显示,开源权重模型与闭源模型的差距比此前测量的更大。文中提到目前还没有出现 Fable 级别的开源模型。作者同时提醒该结果需谨慎看待。

原文 · Ethan Mollick

Artificial Analysis finally moves to closed benchmarks (goodbye GDPval-AA), and finds that open weights models are falling further behind than they had previously measured. Take with a grain of salt, but matches the general vibes that we haven't seen a Fable-class open model.