Summary
Training computer models that understand images and words together usually involves changing the image data in different ways to help the model learn better. However, these usual image changes have started to hit performance limits and can mess up the connection between images and words. The authors of this paper introduced a new way to add small, controlled changes not to the image itself but to the model's internal understanding space. Their method, called Aether, gently mixes in random noise in a balanced way, helping the model get better without losing important details. This approach consistently improved performance across different tasks and models, especially in connecting images and words more reliably than older methods.
What this means in practice
- •For computer vision engineers: Improve training of vision-language models by adding embedding space perturbations that maintain semantic integrity and boost multi-modal alignment.
- •For machine learning platform teams: Integrate embedding-based augmentation techniques to provide more robust and stable training procedures for multimodal AI systems.
Authors
Hyesong Choi, Daeun Kim, Song Park, Taekyung Kim, Byeongho Heo, Sangdoo Yun, Dongbo Min, Dongyoon Han
Abstract
Data augmentation is fundamental to training modern deep vision and multimodal models. While individual methods, such as RandAug, CutMix, Mixup, RandErase, and DropPath, offer strong regularization effects, their combined use has saturated in performance due to overlapping functionalities, and aggressive pixel-level manipulations may disrupt delicate cross-modal alignment. This saturation motivates the search for a new augmentation axis within the embedding space rather than the input space. We introduce Aether, a simple plug-in method that applies diffusion-style random perturbations in the embedding space via controlled alpha-mixing, specifically designed to provide isotropic regularization that remains semantically consistent. Inspired by feature-space perturbations in language models and image degradation in generative pretraining, Aether induces mild yet effective perturbations that smooth the representations without compromising the fine-grained structural information required for strong vision-language encoders. Across diverse architectures and across multiple recognition tasks, Aether delivers consistent gains over the advanced recipe combining CutMix, Mixup, DropPath, and RandAug---a level of improvement rarely observed with modern augmentation alternatives. Notably, Aether demonstrates superior effectiveness in multi-modal alignment, succeeding where traditional pixel-space augmentations fail by providing a stable, isotropic regularization signal that respects the integrity of the high-dimensional feature space.