Idle serving hardware boosts large language model pretraining speed
Accelerating LLM Pretraining with Zeroth-Order Curvature on Idle Serving Compute
Machine LearningArtificial Intelligence
Summary
Training large language models takes a lot of computing power, but some hardware used for running models isn't busy all the time. The authors show that this unused hardware can help speed up training by estimating how the model's learning can improve, even though it can’t calculate exact details needed for normal training. Their method, called ZO-Curv, uses this rough information to help the main training work better, cutting training time almost in half on models with hundreds of millions of parameters.
What this means in practice
- •For machine learning engineers: Use idle inference hardware to accelerate pretraining large language models by integrating curvature estimates into training updates.
- •For cloud infrastructure teams: Reduce resource waste by employing off-peak serving machines to assist in more efficient AI model training workflows.
Authors
Mingchuan Ma, Dongqi Cai, Daqian Yang, Zhenyan Lu, Chen Chen, Teresa Lv, Yan Gao, Shangguang Wang, Mengwei Xu, Nicholas D. Lane, Yu Huang
Abstract
LLM pretraining is increasingly compute-intensive, while substantial inference serving compute remains idle during off-peak periods. However, such serving hardware is difficult to use for conventional first-order training due to its memory, networking and other system requirements. Zeroth-order (ZO) optimization offers a natural interface to this forward-only compute resource, but its gradient estimates break down in pretraining. Unlike fine-tuning, which can confine updates to a small parameter subspace, pretraining must update every parameter, and the alignment of a ZO gradient estimate with the true gradient shrinks as the number of updated parameters grows. We make a different observation: noisy ZO estimates can still provide useful curvature information, even when they are too inaccurate to serve as gradients. Based on this insight, we propose ZO-Curv, which leverages idle forward-only devices to estimate curvature and uses it to adapt and refine the update used during first-order training, thereby accelerating the process. We provide convergence guarantees that characterize the effect of finite-query curvature estimation. Experiments on LLMs from 125M to 1.5B parameters show that ZO-Curv consistently accelerates pretraining. At 770M parameters, ZO-Curv reaches AdamW's 20K-step validation loss in 3.9 hours instead of 7.5 hours, while achieving a lower final loss of 2.70 versus 2.84 under the same step budget. These results show that idle model serving compute can accelerate LLM pretraining when zeroth-order queries are used to estimate curvature rather than gradients directly.