Lenny Rachitsky 推出进阶版 AI 评测系统指南
No coincidence
做 AI 产品的话这篇挺实用：Ramp、Shopify、Cursor 这些公司靠写 evals 把指标提了多少，还附了个免费插件让编码 Agent 帮你干活。
Lenny Rachitsky 联合 Hamel Husain 和 Shreya Shankar 撰写了《Building eval systems》的进阶续篇。文章提到他上周分享的 25 个 PM 职位中近一半要求会写 evals。案例包括 Ramp 将自动收据采集准确率从 35% 提升到 83%，Shopify 的 AI workflow builder 比原前沿模型方案快 2.2 倍、便宜 68%。Harvey 重建 AI 合同审查器后内部质量分接近翻倍，Cursor 的 Auto Balance 路由在成本降低 41% 的同时用户满意度提升。
No coincidence
No coincidence Lenny Rachitsky @lennysan Evals have been coming up more and more in my conversations with podcast guests and PM friends. Nearly half of the 25 awesome PM job openings I shared last week ask for experience writing evals. And leading companies keep sharing what investing in evals bought them: — @tryramp took its automatic receipt collection from 35% to 83% accuracy. — @Shopify shipped an AI workflow builder that's 2.2x faster and 68% cheaper than the frontier-model setup it replaced. — @harvey__ai rebuilt its AI contract reviewer, nearly doubling its internal quality score. — @cursor_ai tuned its Auto Balance routing, with much higher user satisfaction at 41% lower cost. So I asked the 🐐s of evals, @HamelHusain and @sh_reya , to write an advanced sequel to their very popular "Building eval systems that improve your AI product." Drawing on their work with 50+ AI companies, they share the key step most teams skip, what you should (and shouldn't) automate, and a free plugin that lets a coding agent do most of the heavy lifting. Read it here: lennysnewsletter.com/p/advanced-eva… F 🔗 View Quoted Tweet 💬 0 🔄 1 ❤️ 31 👀 5975 📊 6 ⚡
