MLExpert：35B MoE 智能体模型实现自主机器学习工程
MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
一个 35B 的开源方向智能体模型，把 Kaggle 奖牌率从 37.9% 拉到 72.7%，能跟 Kimi-K3、GLM-5.3 掰手腕，做 MLE 自动化的值得看看训练方法。
MLExpert 是一个 35B 参数的 Mixture-of-Experts 智能体模型，通过后训练方法实现长周期自主机器学习工程优化。训练采用 Kaggle-MLE Factory 构建可执行任务，并用分层智能体 SFT 与 Windowed Interaction RL（WIRL）从多轮实验轨迹中学习。在 MLE-Bench Lite 上，其平均奖牌率从基座模型的 37.9% 提升至 72.7%，成绩与 Kimi-K3、GLM-5.3 等更大的前沿模型相当。在 NatureBench Lite、智能体脚手架优化和算法优化任务上的表现显示该优化能力可泛化到其他领域。
MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI). Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback. However, feedback from each experiment reflects several preceding decisions, and a long MLE run contains many such costly experiments. Training must therefore select useful supervision from long interaction trajectories and learn from intermediate outcomes. We propose a post-training method for autonomous long-horizon MLE optimization and apply it to train MLExpert, a 35B Mixture-of-Experts agentic model. We design the Kaggle-MLE Factory to construct executable tasks and an interaction harness for model-directed experimentation. Hierarchical agentic SFT transfers general tool-use and coding capabilities to MLE through outcome-guided supervision. Windowed Interaction RL (WIRL) treats the multi-turn decisions between successive experiment results as an execution window and assigns it a shared outcome-based reward. Asynchronous collection and policy correction support policy updates before the trajectory ends. On MLE-Bench Lite, MLExpert improves the average medal rate of the base model from 37.9% to 72.7%, achieving performance comparable to larger frontier models such as Kimi-K3 and GLM-5.3. Further experiments show that MLExpert improves performance on NatureBench Lite, agent scaffold optimization, and algorithm optimization, suggesting that the learned optimization capability generalizes to other tasks.
