HiPerViT improves texture recognition using texture statistics
HiPerViT: A Hierarchical Perceiver-Vision Transformer Architecture for Multi-Scale Texture Recognition
Computer Vision and Pattern Recognition
Summary
Recognizing textures in images is hard because important details come from how parts of an image relate to each other, not just the shape of objects. The authors designed HiPerViT, a vision transformer that includes special tokens representing these texture details as statistical information. This helps the model better understand textures without needing extra training methods, and it performs better than other vision-only models on multiple texture tests. They found that having explicit texture statistics available is key to the improvement, regardless of the specific model design choices.
What this means in practice
- •For computer vision engineers: Implement improved texture recognition models for applications such as material identification and surface inspection.
- •For industrial quality control teams: Deploy texture-sensitive vision systems to detect subtle surface defects or material variations in manufactured products.$Commercial implications: Enables more accurate automated visual inspection tools that directly leverage texture statistics for defect detection.
Authors
João Pedro C. A. de Sá, Odemir Martinez Bruno
Abstract
Texture recognition remains challenging for modern vision models because discriminative evidence is often carried by higher-order spatial statistics rather than by object shape alone. While Vision Transformers provide strong long-range modeling capacity, their standard object-centric representations do not explicitly expose such statistical structure, which limits texture sensitivity in fine-grained recognition settings. We present HiPerViT, a compact vision-only architecture that injects an explicit second-order statistical prior into a transformer-based recognition pipeline. The method combines global and local image views with a compact bilinear descriptor encoded as a statistical token, and integrates this token with first-order spatial representations through Perceiver-style latent distillation. This design enables direct interaction between spatial tokens and second-order feature co-occurrence statistics, providing the model with explicit access to texture-relevant information without requiring multimodal pretraining or ensemble construction. Across six texture recognition benchmarks, HiPerViT achieves consistent improvements over strong vision-only baselines under the reported evaluation protocols, including gains of +3.05 percentage points on DTD, +10.48 on GTOS-Mobile, and +10.10 on 1200Tex. Beyond benchmark performance, our analyses show that these gains are largely invariant to the backbone depth used to extract second-order statistics and to the ordering of interaction and distillation stages. This pattern suggests that the primary source of improvement is not a specific fusion topology, but the explicit availability of second-order statistical information as a first-class representational signal. These results support explicit statistical tokenization as an effective and robust design principle for texture-centric visual recognition.