Emad Mostaque:AI 模型能效已接近人脑
Qwen 27B 从 60GB 压到 6GB 还保住 95% 性能,居然能跑在树莓派上,这条视频讲了具体怎么做的。
Emad Mostaque 在 Tom Bilyeu 的节目中表示,AI 模型在能效上已基本达到人脑水平。他举的例子是 Prism ML 团队对 Alibaba 的 Qwen 27B 做的压缩:把数值精度从 16 位降到 1.5 位,内存占用从约 60GB 降到 6GB,同时保留 95% 的性能。压缩到 6GB 后,模型可以在 Raspberry Pi 上运行,其功耗与人脑相当。
Emad Mostaque: AI models "have pretty much reached the efficiency of a human brain".
Alibaba's Qwen 27B normally needs about 60GB of memory, and a team from Prism ML cut that to 6GB by storing its numbers with 1.5 bits instead of 16 while keeping 95% of its performance.
At 6GB it runs on a Raspberry Pi, "which uses as much energy as your brain."
---- Full video on "Tom Bilyeu" YouTube channel, (link in comment)