Oscillatory predictive learning improves vision model defenses without extra training effort
Neither Adversarial Training Nor Purification: Emergent Adversarial Robustness from Oscillatory Predictive Learning
Machine LearningArtificial Intelligence
Summary
It is hard to make computer vision systems that don't get fooled by tricky input images called adversarial attacks. Usually, defenses require extra work during training or testing, which slows things down. The authors show that by using special oscillating neuron models and a self-teaching step, the system naturally becomes better at resisting attacks. They achieved strong defense results on popular image datasets without the usual time-consuming steps.
What this means in practice
- •For computer vision engineers: Improve image classification model security against adversarial attacks without costly adversarial training or test-time processing by using oscillatory predictive learning.
- •For machine learning infrastructure teams: Reduce computational overhead during model training and deployment by adopting architectures that promote robustness via inherent inductive biases.
Authors
Mohammed-Yassine Habibi, Klea Ziu, Martin Takáč, Makoto Yamada
Abstract
Adversarial robustness in computer vision is still largely achieved through adversarial training or test-time adversarial purification, both of which introduce significant computational overhead by generating adversarial examples during training or performing iterative denoising at test time. We study whether empirical robustness can instead emerge from architectural and representation-learning inductive biases. We introduce Oscillatory Predictive Learning (OPL), a two-stage framework that combines Artificial Kuramoto Oscillatory Neurons (AKOrN) with predictive self-supervised pretraining using X-PhiNet. Because our default checkpoint uses randomized initial oscillator states, we compare it with other randomized adversarial defense methods that provide precise, reproducible, and strong attack protocols. Experiments on CIFAR-10 and CIFAR-100, with additional corruption evaluation on CIFAR-10-C, demonstrate that our method achieves competitive results under the AutoAttack-rand evaluation protocol. On CIFAR-10 and CIFAR-100, OPL attains 76.63$\pm$0.76$\%$ and 50.44$\%$ robust accuracy, respectively, under $\ell_\infty$, $ε=8/255$, AutoAttack-rand with EoT $K=20$.