NetRL：用 NetGRPO 强化学习优化网络决策
NetRL: Powering Network Decisions via NetGRPO-Driven Reinforcement Learning
一篇把 LLM 用于自适应码率、拥塞控制这类网络任务的研究，NetGRPO 比 NetLLM 高 23.6% QoE，代码和模型权重都开源了。
NetRL 是一个两阶段 LLM 强化学习框架，先用示范轨迹做监督微调，再进行在线策略优化。其核心 NetGRPO 会在多条未来轨迹上评估候选动作，并通过置信度感知预算把更多搜索资源分配给不确定的选项。在基准对比中，NetRL 的 QoE 比 NetLLM 高 23.6%，作业完成时间比 Genet 低 12.2%，网络效用比 Orca 高 24.1%。置信度感知预算把端到端训练时间缩短 27.1%，且验证 QoE 不下降。该框架能稳定收敛，面对未见过的网络轨迹无需重训练即可泛化。
NetRL: Powering Network Decisions via NetGRPO-Driven Reinforcement Learning
Network decision-making tasks, including adaptive bitrate streaming, load balancing, and congestion control, require policies to process heterogeneous states and actions while balancing near- and long-term consequences. Existing solutions generalize poorly across network tasks, require substantial task-specific redesign, and lack reliable evaluation of long- horizon action consequences. Methods that learn through online interaction additionally face unstable optimization. We present NetRL, a two-stage LLM-based reinforcement learning framework that combines supervised fine-tuning on demonstration trajectories with online policy optimization. A universal mixed-embedding interface combines textual task descriptions with precise numerical states, while a network action head produces valid decisions. At its core, NetGRPO evaluates candidate actions over multiple future trajectories, directs deeper search toward uncertain alternatives through confidence-aware budgeting, and performs group-relative, clipped, and KL-regularized updates. NetRL achieves 23.6% higher QoE than NetLLM, 12.2% lower job completion time than Genet, and 24.1% higher network utility than Orca, yielding state-of-the-art performance among the evaluated methods. Confidence-aware budgeting reduces end-to-end training time by 27.1% without reducing validation QoE. NetRL also converges stably and generalizes to unseen network traces without retraining. Code and model weights are available at the repository linked in this paper.
