如果 AI 放缓，万亿资本会把力量投向对齐与机械可解释性
这条推文讲了个反直觉观点：AI 真放缓了，万亿资本不会让实验室闲着，而会全部押注对齐和机械可解释性研究。
作者认为人们把监管放缓想象成官僚小组空转是错的。HFI 之后文化已经转变，AI 实验室和资本市场把说服自己下一次训练运行是安全的列为第一优先级，边际的对齐、安全和模型行为改进现在价值巨大。如果出现放缓，理想情况是把算力窄幅投向机械可解释性、可监控性、决策论、行为泛化这些方向，自动化对齐研究是可行的。作者提醒，一旦恢复安全感、警惕消散，反而是坏发展。
"what would be done during the slowdown" is a good question, because i think people imagine beaurucrat-regulators sitting around on useless subcommittees and wasting the whole thing on wordcel slop analysis. in fact if you asked me two years ago, i would have suggested that is exactly what would happen.
but these are scary times, and they call for us to think lofty & even impractical thoughts, that a 'sensible' person might have dismissed a few quarters ago. crises change the world overnight: the HFI has shifted something about the culture, broadly
this is something obvious that i missed, which is that the ai labs and the markets more generally just have just unbelievable pressure riding on the solving of alignment & control. even now, marginal improvements in alignment, safety, and model behavior are worth tremendous value to the labs. being able to convince themselves that the next training run is safe is the priority zero, one, and two in terms of capabilities progress right now. it's possible that will change and political will dissipates if some sense of comfort re-emerges after a period of relative safety (which would be a bad development; paranoia must be maintained through any potential periods of ai development where the models are too smart to reveal their behavioral predilections under scrutiny)
if there was a slowdown, it would ideally be channeled into some sort of tradeoff in which you can narrowly improve capabilities that prioritize risk-adjusted-return of alignment skills, mechinterp, monitorability, decision theory, behavioral generalization, etc. progress on all these areas is highly possible; automated alignment research is possible. the structure of the businesses may have to change, but all the trillion dollar ai behemoths of the world would shift their focus to doing this and only this at the behest of trillions of dollars in capital markets
