Variance Reduction on the Camera Axis: Multi-View Score Distillation for 3D
2026-06-29 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors improve a method that turns 2D image models into 3D object creators by looking at multiple views at once instead of just one randomly chosen angle. Their approach, called MV-SDI, combines gradients from pairs of views without increasing memory use or changing the original 2D model. This reduces randomness in learning, making the 3D objects more accurate and consistent, and speeds up the process. They show this method works better without needing extra training or multi-view data.
Score DistillationDiffusion Models3D GenerationGradient AggregationMulti-View LearningCLIP ScoreUNetAntithetic SamplingShape ConsistencyGradient Variance
Authors
Marian Lupascu, Mihai Sorin Stupariu, Ionut Mironica
Abstract
Score distillation turns a pretrained 2D diffusion model into a 3D generator, but the per-step gradient is estimated from a single randomly chosen view: it is high-variance and blind to global shape consistency. Prior work addresses this by retraining the diffusion prior on multi-view data; this improves consistency but makes the sampling contribution inseparable from prior quality. We instead isolate the sampling axis. The per-step gradient is one noisy sample of an expectation over views; aggregating K samples per step at a fixed total UNet budget reduces variance without touching the prior. We introduce Multi-View Aggregated Score Distillation (MV-SDI), which aggregates gradients from K views per step via gradient accumulation, keeping peak memory unchanged and the 2D prior frozen, and draws views as antithetic antipodal pairs, a prior-independent geometric property, for balanced angular coverage. At a fixed 10,000-UNet-call budget, K=2 raises CLIP R-Precision from 74.8% to 83.8% and CLIP score from 0.297 to 0.312, with consistent gains on HPSv2 and ImageReward and a 0.0% divergence rate on the 43-prompt benchmark; optimization steps halve as a consequence. K=4 gives a fourfold step reduction at R-Precision 86.9% and CLIP 0.307, still well above the single-view baseline on every alignment metric. MV-SDI is compatible with gradient-based score-distillation pipelines, including Score Distillation via Inversion, and requires no retraining and no multi-view data.