Speaking the Language of Science: Toward a General-Purpose Generative Foundation Model for the Natural Sciences
2026-06-15 • Computation and Language
Computation and Language
AI summaryⓘ
The authors introduce LOGOS, a language model designed to handle many different natural science problems using one shared system. It turns scientific objects and their interactions into sequences of tokens, allowing the model to understand complex relationships without needing explicit 3D data. LOGOS can perform well on a variety of tasks by predicting the next token in these sequences, and larger versions of the model perform better. The authors suggest that future science AI might best develop by combining large language models with specialized scientific training, and they have made their model publicly available for others to use.
generative language modelautoregressive frameworktoken sequencesscientific grammarnext-token predictionmulti-domain pre-trainingstructural interactionsnatural sciencesmodel scalingAI for Science (AI4S)
Authors
Mingyang Li, Yurou Liu, Jieping Ye, Bing Su, Ji-Rong Wen, Zheng Wang
Abstract
In this report, we present LOGOS (Language Of Generative Objects in Science), a scientific generative language model that unifies heterogeneous tasks across the natural sciences within a single autoregressive framework based on a shared scientific grammar. It encodes diverse scientific objects and their spatial interactions as token sequences over a common vocabulary. By representing spatial contact and constraint patterns as discrete tokens, the model captures complex structural interactions in a purely sequential manner, without relying on explicit coordinates or geometric neural networks. This unified representation enables a wide range of downstream tasks to be formulated consistently as next-token prediction in the same grammar space, creating strong alignment between continued multi-domain pre-training and downstream objectives. Across diverse tasks, LOGOS consistently matches or outperforms domain-specific baselines, providing preliminary evidence for the feasibility of "one model fits all" in the natural sciences. We train LOGOS models at different scales (1B, 3B, and 8B parameters) and find a consistent positive correlation between model size and performance. This suggests that the future of AI for Science (AI4S) may not lie in building an independent technical stack that is separated from large language models (LLMs). Instead, it may depend on deeply aligning scientific foundation models with LLMs through shared architectures, shared training paradigms, and shared inference infrastructure, so that LLMs can truly become a new entry point for AI4S. We release the model weights and associated resources to facilitate further research.