LongChang 基准测试：LLM 能否自主发现被修改的物理定律
Can LLMs Uncover Hidden Physical Laws through Free Exploration?
让模型在一个重力规律被改掉的世界里自己发现异常，12 个模型只有 GPT-6-Astra 做到一次，其余全挂，测试设计挺有意思。
arXiv 论文提出 LongChang 框架，自动生成物理定律被修改的模拟世界，让 LLM 在无任务提示的情况下自主做实验发现异常规律。12 个受测 LLM 中，GPT-6-Astra 的定律发现率为 16.7%，其余 11 个均为 0%。探索记录显示模型普遍未能调查所有异常现象，对被修改定律的识别也不完整。
Can LLMs Uncover Hidden Physical Laws through Free Exploration?
Scientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more slowly, and discover the law through experiments without being told what to investigate? To evaluate this ability, we introduce LongChang, an automated framework that creates simulated worlds with altered physics, where one or more laws differ from ordinary physics. The evaluated LLMs act as explorers, manipulating objects and building tools to investigate their own questions without research tasks or hints about these laws. Separate LLM agents generate and review scenes, review tool designs, and evaluate reports for correctness and experimental support using the altered physical laws and exploration records. These records guide scene evolution for further exploration, allowing the full cycle to run without human intervention. Among 12 LLMs, GPT-6-Astra achieves a discovery rate of 16.7%, while the other 11 score 0%. The exploration records reveal uninvestigated anomalies and only partial identification of the altered physical laws.