Large language models fine-tuned to write verified particle physics code
LLM-Based FORM Code Generation with Verification-Driven Fine-Tuning
Computational Engineering, Finance, and ScienceComputation and Language
Summary
Writing code in a special language called FORM is important for particle physics but hard for AI to do correctly out of the box. The researchers found big AI language models could not run FORM code successfully without help. They created a method that uses the FORM program itself to generate and check examples, then trained a smaller AI model on these examples. This fine-tuned model can write verified FORM code much better than larger models and still keeps good general coding skills.
What this means in practice
- •For scientific software developers: Build AI-assisted tools that generate correct FORM code for complex particle physics computations to speed up theoretical research workflows.
- •For computational physicists: Use AI to produce verified FORM scripts for multi-loop Feynman diagram calculations, reducing manual coding errors and development time.
Authors
Bakar Chargeishvili
Abstract
FORM is a domain-specific symbolic manipulation language widely used in particle physics for processing the very large algebraic expressions arising from multi-loop Feynman diagram calculations. Despite its central role in precision theoretical physics, no artificial-intelligence tooling exists, to our knowledge, for assisting physicists in writing FORM code. We show that contemporary large language models (LLMs), including frontier models with hundreds of billions of parameters, achieve a zero-percent execution pass rate on our instruction-following and tutorial-style FORM tasks without documentation in a single attempt, establishing FORM as a genuine zero-shot language for LLMs at the time of writing. We then present a verification-driven data generation pipeline that uses the FORM binary itself as an execution oracle to produce and validate a corpus of 4,633 training examples spanning deterministic computations, open-ended programs, tutorial code, and knowledge question-answer pairs. Fine-tuning a compact open-weights model (Qwen3-8B) with quantized low-rank adaptation (QLoRA) yields a specialist that, evaluated on four complementary benchmarks (840 tasks, single attempt each), decisively outperforms frontier models with up to 756B parameters in execution rate and in strict, FORM-verified output matching on the larger benchmarks, and remains statistically indistinguishable from them on the smaller, harder ones. General reasoning and coding capabilities are preserved within 2.6 percentage points.