Fireworks AI:路由让 113 个编码任务解题率达 97.6%
We ran 18 models across 113 real coding tasks on DeepSWE, then went back and asked a simple question...
Fireworks AI 测了 18 个模型:把任务路由给最擅长的那个,比单用最强模型更准也更省。
Fireworks AI 在 DeepSWE 上用 113 个真实编码任务评测了 18 个模型。单一最强模型的解题率为 74.1%,每任务成本 6.52 美元。把每个任务路由给最擅长的模型后,整体解题率升至 97.6%,每任务成本降至 1.88 美元。Fireworks AI 在博客中称,模型路由是下一个前沿方向。
We ran 18 models across 113 real coding tasks on DeepSWE, then went back and asked a simple question...
We ran 18 models across 113 real coding tasks on DeepSWE, then went back and asked a simple question: what if every task had routed to the model that handled it best? Answer: 97.6% solve rate at $1.88 per task, versus the best model at 74.1% at $6.52. The next frontier is a router. Full analysis: fireworks.ai/blog/the-front… 💬 0 🔄 1 ❤️ 1 👀 201 📊 1 ⚡