HiResNets: Native Full-HD Video Recognition with Foveal Residual Streams
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionPerformance
AI summaryⓘ
The authors developed a new type of neural network called HiResNets that can handle very high-resolution video more efficiently than traditional models. Instead of processing the whole image at once, their method focuses on parts of the image with high detail using a special log-polar transformation, similar to how human eyes focus on different spots to build a complete picture. This approach reduces memory and computation needs, helping the model recognize small objects and fine details better in egocentric video tasks. Their experiments show these networks mimic human-like focusing behavior and improve video recognition performance.
residual networksconvolutional blockslog-polar transformhigh-resolution videoegocentric video recognitionfoveationquadratic memory growthvision transformersimage resolutionspatial attention
Authors
Shivani Mall, Swarnim Jain, Joao F. Henriques
Abstract
Much of the recent progress in image and video recognition has come at the cost of memory: larger models, increased resolution, and longer temporal contexts. An inevitable component is the quadratic (or larger) growth of memory and compute based on image resolution, which is a property of the grid sampling used in convolutional networks and vision transformers. In this work we study residual networks whose convolutional blocks have logarithmic-square growth instead, enabling them to process very high-resolution video quickly. The key insight is to use a residual architecture's residual stream as a high-resolution buffer, to which convolutional blocks only read and write via log-polar image warp operations. Layers adaptively focus on different parts of each frame, with very high resolution only near the focus point. A complete high-resolution representation is built up in the residual stream, analogous to eye saccades creating a complete picture in biological vision, and a theoretical construction is presented that eliminates the quadratic dependency of the residual stream resolution. Experiments demonstrate that our proposed HiResNets learn to foveate around scenes similarly to human vision, and have superior performance in difficult egocentric video recognition tasks, especially egocentric video with small objects and fine-grained recognition.