OpenDocRouter 为 LightOnOCR-3-1B 等开源 OCR 模型加入投机解码加速
LlamaIndex 给 OpenDocRouter 上的 OCR 模型加了投机解码，LightOnOCR-3-1B 跑得更快了，做文档识别的可以去测速。
LlamaIndex 团队为 OpenDocRouter 上的开源 OCR 模型训练了新的投机解码器，用于降低推理延迟。LightOnOCR-3-1B 等模型的推理速度有明显提升。团队分别测量了单请求（最好情况）和 64 请求并发下的每秒 token 数。TeleOCR 在高并发下加速较少，原因是它调用次数多但每次解码量小。
The OSS models on OpenDocRouter are slowly powering up! 🚀 We trained additional speculative decoders to help with latency. You should see considerably faster inference times using models like @LightOnIO LightOnOCR-3-1B! We measured tokens per second across single requests (best case) and 64 requests (saturated throughput). TeleOCR sees less speedup at scale since it makes a lot of tiny calls, where decoding latency matters a bit less. Try these models out today! opendocrouter.ai C 💬 5 🔄 1 ❤️ 7 👀 1222 📊 6 ⚡
