NemotronLabs 开源全双工语音模型 VoiceChat，支持工具调用
NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
NemotronLabs 开源了全双工语音模型 VoiceChat，能边聊边调工具，被打断也接得住，做语音智能体的可以看看。
NemotronLabs 在 arXiv 发布开源全双工语音对话模型 VoiceChat，可在同一流式架构内完成听音、转写、推理、调用工具和说话。模型由流式语音编码器、decoder-only 语言模型、用于增量转写的 RNN-T 分支和流式 TTS 解码器组成，并为智能体文本与结构化函数调用设置并行输出流。在 Full-Duplex-Bench 1.0 上，其停顿处理接管率在参评开源权重系统中最低，用户打断后 100% 实现接管，打断后响应质量得分 4.33/5。在 Full-Duplex-Bench 1.5 上，93% 的用户插话后能恢复应答；VoiceBench 归一化平均分 55.1，Full-Duplex-Bench 3.0 工具选择 F1 达 82.5%，参数准确率与端到端工具执行仍待改进。
NemotronLabs VoiceChat: An Open Full-duplex Speech-to-Speech Model with Tool Calling Capabilities
We introduce NemotronLabs VoiceChat, an open full-duplex speech-to-speech model with native tool-calling capabilities. NemotronLabs VoiceChat combines a streaming speech encoder and decoder-only language model with parallel specialized output streams for agent text and structured function calls, an auxiliary RNN-T branch for incremental user transcription, and a streaming TTS decoder. This design enables the model to listen, transcribe, reason, invoke tools, and speak within a unified streaming architecture while preserving the temporal behavior required for natural conversation. On Full-Duplex-Bench 1.0, NemotronLabs VoiceChat achieves the lowest pause-handling takeover rates among evaluated open-weight systems, 100\% takeover following user interruptions, and a 4.33/5 post-interruption response-quality score. On Full-Duplex-Bench 1.5, it resumes its response after user backchannels in 93\% of cases. NemotronLabs VoiceChat obtains a 55.1 normalized average on VoiceBench and, on Full-Duplex-Bench 3.0 (FDB 3.0), achieves 82.5\% tool-selection F1, while argument accuracy and end-to-end tool execution remain areas for improvement. These results demonstrate that full-duplex interaction, speech recognition and generation, general language capabilities, and external tool use can be integrated in a single open speech-to-speech model without sacrificing real-time conversational behavior.
