Fly visual system model shows efficient but limited robustness to attacks
Measuring Robustness and Efficiency in a Connectome-Constrained Fly Visual System Model: Linking an Adversarial and a Perceptual Illusion
Computer Vision and Pattern Recognition
Summary
Biological vision systems like a fly's are energy efficient and handle visual tasks well despite limited wiring. This paper studies a model of the fly's visual system (FlyNet) and compares it to artificial neural networks on a collision detection task. They found FlyNet uses far fewer computations and is more robust to some common video noises than certain artificial models, but it is vulnerable to deliberate adversarial attacks. The study links failure cases in the model to similar illusions in biological vision, suggesting that brain-like wiring helps efficiency more than overall robustness.
What this means in practice
- •For autonomous vehicle developers: Improve collision detection models by integrating efficient, connectome-inspired architectures to reduce computational cost while maintaining robustness against common video distortions.
- •For computer vision engineers: Design neural networks with recurrence and temporal context to enhance robustness against video quality degradation and adversarial perturbations.
Authors
Alexandre Garcia-Duran, Alejandro Rodriguez-Garcia, Manuel Molano-Mazón, Alexandre Hyafil, Srikanth Ramaswamy, Janne K. Lappalainen
Abstract
Biological visual systems compute reliably under tight metabolic and wiring constraints, whereas artificial vision systems consume far more energy and can be sensitive to adversarial perturbations. Whether networks with brain-like wiring are more robust at lower computational cost is unclear. We address this by benchmarking a task-optimized, continuous-time recurrent neural network of the fly visual system constrained by its synaptic connectome (FlyNet) against artificial neural networks. We compare it against CNNs, RAFT, and continuous-time recurrent neural networks without detailed connectome constraints (sCTRNN, CTRNN) on a collision-detection task within a differentiable 3D rendering pipeline. FlyNet is adversarially vulnerable but, relative to its compute, more robust to common video corruptions than RAFT, while using up to five orders of magnitude fewer FLOPs. The CTRNNs remain more robust than FlyNet across almost all tested perturbations, suggesting that connectome constraints benefit efficiency, through their low edge count, over robustness. Recurrence improves robustness to white-box attacks in RAFT and temporal context improves robustness to common corruptions. FlyNet's position and velocity representations degrade under adversarial textures but recover under counterfactual textures that restore correct classification while staying equally far from training, ruling out unfamiliar textures as the cause of failure. The reverse-phi illusion similarly disrupts these representations and inverts the same low-level motion statistic as the adversarial textures. Optimizing this input statistic alone, without access to the network, suffices to induce classification failure, attributing both perturbations to the same circuit vulnerability. Connectome-constrained models can thus surface adversarial failure mechanisms as testable hypotheses about biological perceptual vulnerabilities.