Flyvis reveals limits in testing fly brain vision models
FlyHack: Stress-Testing Mechanistic Claims in a Connectome-Constrained Visual Model
Computer Vision and Pattern Recognition
Summary
Understanding how the fly brain processes motion is challenging, and scientists build computer models based on the fly's brain wiring. The authors examined many versions of one such model and compared them to simpler neural network models to see how reliable their conclusions about brain function were. They found that the model's predictions and explanations can change depending on how the model is tested and how it's built, meaning claims about the fly's vision system must be made carefully. This work suggests these brain-based models are useful for generating ideas but that their details depend a lot on the model setup.
What this means in practice
- •For computational neuroscientists: Use connectome-constrained models to test hypotheses about neural circuits with awareness of model-dependent limitations.
- •For machine learning engineers: Compare biological and task-optimized CNN models to evaluate robustness of mechanistic claims in vision modeling.
Authors
Aashish Dhawan
Abstract
Connectome-constrained neural networks make biological components directly accessible to intervention, but their mechanistic conclusions can still depend on the comparison model, intervention, circuit representation and trained solution. We stress-test these dependencies in flyvis, a \emph{Drosophila} motion-vision model, using its 50-model ensemble, task-matched causal CNN controls and replicated interventions. At the level of model behaviour, three-level reverse-phi reversal in flyvis is strongly period- and velocity-dependent and also appears in task-matched CNNs; neither family reproduces the fly's stationary-sawtooth response. Adversarial direction reversal requires a perturbation bound of 0.21--0.27 times stimulus contrast in flyvis versus 0.13--0.25 in CNNs, but the gap over a parameter-matched CNN largely closes for larger CNNs. At the level of mechanism, matching attribution to the intervention matters: for held-out-mean replacement, first-order edge attribution patching reaches pooled AUROC 0.970 and Spearman 0.872, versus 0.933 and 0.126 when its zero-aligned score is evaluated against the same ground truth. Circuit conclusions also vary with representation and ensemble member. Greedy cell-type sets meet a model-relative T4/T5 tuning criterion in 25 of 50 models, while T4/T5 output silencing reduces the tested optic-flow behaviour to at most two of eight clips in 40. In six models where the cell-type search first succeeds, the corresponding connection sets can be pruned to 13--31 added classes; under an additional two-sided response-amplitude check, only two of those six first-success cell-type sets still meet the tuning threshold. These results support a narrower use of connectome-constrained models: they make mechanistic hypotheses directly testable, but the resulting claims must be scoped to the model family, intervention, representation and trained solution.