MiniCPM-V 4.7 在 iPhone 上实现 20.51 tokens/秒本地多模态推理
OpenBMB 把 35B 多模态模型塞进 iPhone，每秒 20+ token，比基线快 6 倍，代码开源可以直接上手玩。
OpenBMB 发布 MiniCPM-V 4.7 的端侧优化版本，35B 多模态模型在 iPhone 18 Pro Max（12GB）上达到平均 20.51 tokens/秒的解码速度。相比同设备未优化的 SSD 专家加载基线，速度提升 6.07 倍。加速来自 Lookahead Expert Scheduling（LES）、优化专家加载、细粒度量化和 DSpark 推测解码。代码已在 GitHub 和 Hugging Face 开放。
35B multimodal AI. Running locally on an iPhone. Now at over 20 tokens/sec. 🚀
MiniCPM-V 4.7 achieves 20.51 tokens/sec average decoding speed on iPhone 18 Pro Max (12GB) — a 6.07× speedup over the same device’s unoptimized SSD expert-loading baseline.
⚡ Behind the speed: Lookahead Expert Scheduling (LES), optimized expert loading, fine-grained quantization, and DSpark speculative decoding.
📱 Watch the new on-device demo below. Powerful multimodal AI, right in your pocket.
Try it 👇 🔗 GitHub: https://t.co/msjm2QkTGp 🤗 Hugging Face: https://t.co/W95QKGh0wY LES: https://t.co/6Ul7qXBItU
#MiniCPM #OpenBMB
