arXiv论文提出迭代未对齐法，估算智能体罕见事件概率效率提升800倍
Rare Event Estimation via Iterative Unalignment
智能体部署前想知道灾难事件发生概率有多低？这套开源方法比蒙特卡洛快800倍，代码可直接跑。
arXiv论文提出'迭代未对齐'方法，通过微调原模型权重构造重要性采样的提议分布，用梯度搜索定位能触发目标事件的模型变体。在约120M和2.6B两个规模的模型上测试，覆盖3个事件族、300多个概率低至10⁻⁹的罕见事件，参考概率的相对标准误差控制在10%以内。对概率低于10⁻⁷的事件，该估计器的计算加权效率比朴素蒙特卡洛高出800倍以上。代码已在GitHub开源，适合在部署前量化智能体自主行为中灾难性事件的发生概率。
Rare Event Estimation via Iterative Unalignment
As agents are deployed with increased autonomy, even extremely rare events along their stochastic output trajectories can occur and prove catastrophic. Safe deployment therefore does not depend on whether these events can occur, but on how often they might. We study the problem of estimating the probability of rare events that arise from stochastic variation in the agent's own actions. Estimating this type of risk requires searching over the combinatorially vast space of trajectories. Naive Monte Carlo is computationally prohibitive in this regime, and constructing effective importance sampling (IS) proposals requires coordinated changes to a context-dependent chain of conditional distributions. We develop a new IS method that perturbs the original model's weights to construct the proposal. The proposal is itself a differentiably parameterized language model, enabling gradient-based search over weight space. We formulate an objective that combines a differentiable surrogate for event amplification and an adaptive regularization scheme that dynamically balances amplification against estimator stability. We evaluate our approach on $\sim$120M and $\sim$2.6B models across three event families spanning 300+ rare events as rare as $10^{-9}$, with reference probabilities computed with $<10\%$ relative standard error. In our most verifiable settings, we observe that our IS estimator achieves over $800\times$ compute-weighted efficiency gains over naive Monte Carlo for events with probabilities lower than $10^{-7}$. Our implementation is available at https://github.com/namkoong-lab/iterative-unalignment.
