Large language model makes some discoveries in altered physics worlds
Can LLMs Uncover Hidden Physical Laws through Free Exploration?
Computation and LanguageArtificial IntelligenceMultiagent Systems
Summary
Sometimes in science, we find new rules by noticing something strange and exploring it. The authors tested if large language models (LLMs) could investigate made-up worlds where the usual laws of physics were changed, without any instructions. They built a system called LongChang that lets LLMs experiment and make tools on their own. Only one LLM, GPT-6-Astra, found the hidden new laws some of the time, while others did not. This shows that LLMs can sometimes uncover unusual physics but often miss clues or only partially solve the puzzle.
What this means in practice
- •For simulation platform developers: Create AI agents that autonomously explore and test altered physics in virtual environments for research and teaching simulations.
- •For game designers: Use LLM-driven agents to discover and adapt to new game physics rules without direct programming of those rules.
Authors
Zhen Yang, Wen Wang, Xander Xu, Ying-Cong Chen
Abstract
Scientific discovery often begins with an unexpected observation that prompts a new research question. In a world with altered physical laws, can large language models (LLMs) notice unexpected behavior, such as heavier objects falling more slowly, and discover the law through experiments without being told what to investigate? To evaluate this ability, we introduce LongChang, an automated framework that creates simulated worlds with altered physics, where one or more laws differ from ordinary physics. The evaluated LLMs act as explorers, manipulating objects and building tools to investigate their own questions without research tasks or hints about these laws. Separate LLM agents generate and review scenes, review tool designs, and evaluate reports for correctness and experimental support using the altered physical laws and exploration records. These records guide scene evolution for further exploration, allowing the full cycle to run without human intervention. Among 12 LLMs, GPT-6-Astra achieves a discovery rate of 16.7%, while the other 11 score 0%. The exploration records reveal uninvestigated anomalies and only partial identification of the altered physical laws.