OpenAI 2016年的智能体项目回顾，十年后Astra成功解决计算机使用问题，GPT-6终于能可靠预订航班。
2016年OpenAI的World of Bits项目让智能体通过观察屏幕像素、移动鼠标来预订联合航空航班。该项目使用Pascal Titan X GPU和手工设计的奖励函数进行强化学习。十年后，Astra在同一宇宙中重生，解决了计算机使用智能体问题。GPT-6现在可以可靠地完成预订任务。
Good old days at OpenAI in 2016: an agent stares at screen pixels, moves a mouse, and books a flight...
Good old days at OpenAI in 2016: an agent stares at screen pixels, moves a mouse, and books a flight on United. We called it World of Bits, inside OpenAI Universe. 10 yrs later, Astra is reincarnated in the same universe. Even the naming is astronomically correct 😆 Universe was perhaps the most ambitious AI infra project at the time, but we couldn't quite figure out how to solve it. A policy with zero prior knowledge of what a "submit" button does has to rediscover the entire internet visual lingua by trial and error. In retrospect, RL from scratch against hand-drawn, per-task "artisan" reward functions on a bunch of Pascal Titan X GPUs was completely doomed. To solve computer use agent, the right way turns out to be boiling the ocean first (hillclimb on every general task you can find), and then specialize back down to the screen pixels and keystrokes. Or simply, a "Specialized Generalist". Lessons learned: one step ahead of everyone, you're a pioneer. Three steps ahead, you're a prophet. Five steps ahead, you're a martyr. Congrats, GPT-6! That United flight finally gets booked, reliably this time. The 2016 intern in me has a big smile. Your browser does not support the video tag. 🔗 View on Twitter 💬 3 🔄 0 ❤️ 23 👀 1708 📊 5 ⚡