Framework connects model capabilities with training and compute resources
The Capability Manifold and ML Scaling Laws
Machine LearningArtificial Intelligence
Summary
Machine learning models are usually judged by how well they predict data, but this doesn't always show how good they are at things like planning or reasoning. The authors introduce a way to map a model's capabilities across different stages, like before and after training and while it's running, using a concept called a capability manifold. This framework helps understand how changes in computing power, model size, and data affect what a model can do. They show how existing rules that predict performance fit into this new map.
What this means in practice
- •For ml system engineers: Optimize resource allocation across training and deployment to achieve targeted model capabilities.
- •For cloud infrastructure teams: Plan compute and data provisioning strategies using unified capability trajectories to meet client performance goals.
A theory result. No direct application yet.
Authors
Syed Ali Raza Zaidi, Maryam Hafeez
Abstract
Existing machine learning (ML) scaling laws relate predictive loss to compute, model parameters, and data. However, as models are increasingly deployed through agentic harnesses, loss alone is insufficient to characterize downstream performance: models with similar loss can exhibit different capabilities in reasoning, retrieval, planning, and adaptation. Yet, no unified framework connects such capabilities to the coupled resources available across the ML lifecycle. We bridge this gap by introducing a capability manifold, a multidimensional framework mapping downstream capabilities to pre-training, post-training, and test-time resources through bounded scaling functions. Analytical Jacobians quantify capability sensitivity to resource changes and interactions. As an initial application, we embed Kaplan- and Chinchilla-type scaling laws and test-time compute within the framework, demonstrating how existing scaling relationships can be unified as trajectories on a common capability manifold.