Deepgram 将 Nova-3 语音模型迁移至 AWS Trainium2 优化推理
How Deepgram optimized voice AI at scale with AWS Trainium
精选理由
Deepgram 把自家 Nova-3 语音模型搬到 AWS Trainium2 芯片上跑，讲了移植细节、kernel 调优和成本数据，做语音推理的可以看看。
Deepgram 与 AWS 前沿部署工程团队合作，把 Nova-3 语音转文字模型迁移到 AWS Trainium2 芯片做离线批量推理。移植过程基于 AWS Neuron 软件栈，并进行 kernel 级调优。团队还用 Rust 写了数据并行运行时来支撑批量转录。文章给出了在批量转录场景下的吞吐量和成本结果。
原文 · AWS Machine Learning Blog
How Deepgram optimized voice AI at scale with AWS Trainium
Deepgram worked with the AWS Forward Deployed Engineering team to port its Nova-3 speech-to-text model to AWS Trainium2 for offline batch inference. This post covers porting to the AWS Neuron stack, kernel-level tuning, a data-parallel Rust runtime, and the throughput and cost results on batch transcription.
