Stacking different vision models improves accuracy and reliability
CORE-STACK+: Meta-Learning for Deep Stacked Generalization
Machine LearningComputer Vision and Pattern Recognition
Summary
Combining multiple types of deep learning vision models often improves their overall predictions, but current ways of mixing them can lead to unstable or badly calibrated results. The authors propose a new method called CORE-STACK+ that carefully selects and weights models with advanced math tools to avoid these problems. This approach improves prediction accuracy, reduces computation, and produces better confidence estimates. They tested this on popular image benchmarks and demonstrated consistent gains and efficiency.
What this means in practice
- •For computer vision engineers: Build more accurate and calibrated vision ensembles by selecting diverse models and blending them with CORE-STACK+ to improve predictions on large-scale image tasks.
- •For machine learning platform teams: Integrate a preconditioning pipeline to automate ensemble selection and weighting, reducing inference costs while maintaining or boosting model performance.
Authors
Noor Islam S. Mohammad
Abstract
Stacking heterogeneous vision backbones (CNNs, ViTs, and hybrids) is the de facto recipe for accuracy, calibration, and robustness, yet two coupled pathologies limit its returns. Prediction-space multicollinearity ill-conditions the meta-learner's Gram matrix, inflating weight variance and producing brittle solutions on a thin manifold. Calibration collapse compounds constituent miscalibration through naive linear stacking, so adding more models can hurt expected calibration error (ECE). Existing remedies, ridge regularization, greedy selection, model soups, and SWAG address at most one of these issues, and none jointly target conditioning and calibration in heterogeneous prediction pools. We introduce CORE-STACK+, a preconditioning pipeline with four components: (i) a kernelized redundancy filter that removes non-linear inter-model dependencies invisible to Pearson correlation, using Centered Kernel Alignment (CKA) [23]; (ii) a $<15$K-parameter differentiable meta-feature gate that learns per-sample attention over ensemble statistics; (iii) a spectrum-adaptive Ridge penalty $lambda^{star}=lmax(Chat)/SNR(Chat)$ derived from a Marchenko-Pastur signal-noise decomposition, eliminating nested cross-validation; and (iv) a Laplace-approximate Bayesian blender replacing inverse-RMSE heuristics. We prove a PAC-Bayes excess-risk bound that, for the first time, jointly accounts for prediction-space redundancy and meta-learner capacity. Across six benchmarks, CORE-STACK+ delivers $+1.8\%$ top-1 on ImageNet-1K, $-4.2$ mCE on ImageNet-C, $+0.9$ mIoU on ADE20K, and $+1.3$ AP on COCO, while reducing retained models by 35-57% and inference FLOPs by up to $41%$. ECE improves $2.1\times$ over deep ensembles without post hoc temperature scaling.