SkillScaffold：按需注入技能的智能体强化学习训练框架
SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection
智能体 RL 训练按需注入文本技能，省掉逐个试错的成本，ALFWorld 等四个基准全超基线，代码开源。
arXiv 论文提出 SkillScaffold，用低成本的技能效用估计替代依赖大量环境 rollout 的评估方式来训练智能体。方法复用已验证结果的任务轨迹生成候选技能，通过结果感知的似然比较估算效用，无需为每个候选技能单独试错。效用估计决定是否注入技能及注入哪一个，没有候选被判定有效时保持无技能执行。在 ALFWorld、WebShop、ScienceWorld 和 τ²-Bench 四个基准上，该方法超过强智能体 RL 基线与基于技能的 RL 基线，覆盖短时程与长时程任务。消融实验支持基于效用的选择与弃用机制，代码已开源在 GitHub。
SkillScaffold: Scaffolding Agentic Reinforcement Learning with Policy-Adaptive Skill Credit Assignment and Selective Injection
Agentic reinforcement learning (RL) enables large language models (LLMs) to learn from multi-turn interactions with verifiable outcomes. Textual skills can improve exploration by distilling prior interactions into reusable guidance. However, existing skill-based RL approaches often either assume skill injection is beneficial or rely on indirect feedback or rollout-intensive evaluation to assess its utility. Our pilot studies show that rollout-based utility estimation for skill credit assignment is costly and unstable under limited sampling, while skill benefits vary across candidates and depend on the current policy's competence on each query. To address these challenges, we introduce SkillScaffold, a policy-adaptive training scaffold that combines low-cost skill-utility estimation with selective skill injection. By reusing outcome-verified training trajectories, SkillScaffold generates candidate skills and estimates their utility through outcome-aware likelihood comparisons, without candidate-specific environment rollouts. The resulting utility guides whether and which skill to inject, retaining skill-free execution when no candidate is estimated to help. Joint training enables the Generator to produce skills tailored to the evolving Executor. Scaffolded RL encourages skill internalization for skill-free inference. Experiments on ALFWorld, WebShop, ScienceWorld, and $τ^2$-Bench demonstrate gains over strong agentic-RL and skill-based RL baselines across both short- and long-horizon tasks. Ablations and further analyses support the importance of utility-based selection and abstention. Our code is available at https://github.com/Vaidurya00/SkillScaffold.
