Japanese large language models prioritize transparency and usability
LLM-jp-4 Technical Report
Computation and LanguageArtificial Intelligence
Summary
This paper introduces LLM-jp-4, a set of large language models designed specifically for the Japanese language. The models are made with a focus on being open and reproducible, using a huge amount of training text to improve performance. The team shares clear steps on how they chose data and software, helping others verify and build on their work. Their new models perform better than earlier versions and as well as similar open models.
What this means in practice
- •For software developers: Integrate high-performing, transparent Japanese language models into natural language applications requiring reliable and reproducible results.
- •For language technology companies: Develop Japanese AI tools and services using the documented LLM-jp-4 models to ensure transparency and competitive performance.$Commercial implications: Enables building and selling AI-based Japanese language services that rely on verifiable, open development processes.
Authors
LLM-jp, :, Chaoran Liu, Daisuke Kawahara, Hideyuki Tachibana, Hirokazu Kiyomaru, Jun Suzuki, Koshi Eguchi, Kouta Nakayama, Qianying Liu, Rio Yokota, Ryo Takahashi, Sadao Kurohashi, Sosuke Hosokawa, Taishi Nakamura, Takashi Kodama, Yun-Ang Wu, Yusuke Miyao, Yusuke Oda
Abstract
This paper presents LLM-jp-4, a family of Japanese sovereign large language models (LLMs) developed with an emphasis on transparency, reproducibility, and practical usability. LLM-jp-4 comprises an 8B dense model and a 32B-A3B mixture-of-experts model, both trained from scratch on approximately 12 trillion tokens. To support independent verification and reproduction, we establish explicit criteria for selecting training data, external models, and software, and make the development process publicly available. Evaluation results demonstrate substantial improvements over previous LLM-jp models and competitive performance against open-weight models of similar scale. By documenting the development principles, training procedures, and empirical findings of LLM-jp-4, we aim to provide a reference for future sovereign AI development.