Convolutional networks learn shape better by ignoring textures
Mitigating Shortcut Learning: Texture-Penalized Prototype Networks
Computer Vision and Pattern Recognition
Summary
Standard image classification networks often rely too much on textures, which can cause errors when textures change but shapes remain the same. To fix this, the authors designed a new network structure that penalizes the use of texture cues, encouraging the model to focus on the overall shape of objects instead. This approach helps the network recognize objects more accurately even when textures are misleading or altered. Importantly, this method works well without needing large or specially modified training datasets.
What this means in practice
- •For computer vision engineers: Improve object recognition systems to maintain accuracy despite texture changes by incorporating texture-penalized prototype networks.
- •For autonomous vehicle developers: Enhance perception modules to better identify objects in varying environmental conditions by reducing texture dependence in visual models.
Authors
Akshay Anilkumar Girija, Elena Hoemann, Frank Köster, Sven Hallerbach
Abstract
Standard Convolutional Neural Networks (CNNs) exhibit severe performance degradation due to a strong inductive texture bias that prioritizes local, high-frequency patterns over global structural shapes. This dependency causes confident misclassifications during textural changes or environmental effects. To address this flaw, this study introduces the Texture-Penalized Prototype Network (TPPN), a novel architectural framework that shifts this inherent bias without depending on resource-intensive augmented datasets. Specifically, a Texture-Penalization Branch (TPB) imposes a penalty to suppress the extraction of local texture proxies, forcing the network backbone to discard high-frequency cues and extract purified, shape-biased representations. By evaluating similarities within a prototype-based hypersphere derived from the final convolutional features, the approach enforces strict geometric constraints, treating objects as compositions of essential parts to achieve robust classification. Evaluations on texture-shape cue-conflict datasets and synthetic noise benchmarks demonstrate the stronger shape bias of this structural disentanglement. The proposed framework reduces the inherent texture bias of a baseline ResNet-50 from 55.11% to 29.73%, surpassing the texture-suppression capabilities of an off-the-shelf Vision Transformer (ViT-B/16). Furthermore, the approach demonstrates robust generalization under cue-conflict conditions, resisting textural shortcut learning when encountering Out-of-Distribution (OOD) shapes. The model maintains stronger shape accuracy against elevated perturbations. On clean validation data, the architecture incurs a minimal drop in accuracy of 0.90 percentage points. This provides a structural, efficient solution to CNN texture bias.