Tissue graph encoder improves medical image analysis accuracy and efficiency
TAN: A Size- and Form-Adaptive Masked-Convolution Encoder for Tissue Graphs
Computer Vision and Pattern Recognition
Summary
Analyzing tissue images is easier when the regions are kept whole, not chopped into squares. The authors show that current methods can accidentally include background information that changes results depending on arbitrary settings. They introduce TAN, a new small computer model that looks at each tissue region exactly as it is, ignoring surrounding space, which stops background from confusing the analysis. TAN works as well as much bigger models but uses far less computing power and no pretraining.
What this means in practice
- •For medical image analysts: Improve accuracy and reliability of tissue classification in histology images by using a model that ignores background artifacts during encoding.
- •For healthcare ai developers: Build efficient, compact tissue image encoders that perform comparably to large models without requiring extensive pretraining.
Authors
Jan Herschel, Anselm von Gladiss
Abstract
Tissue graphs represent histology images as graphs over tissue-conforming superpixels, which keeps homogeneous tissue regions intact instead of cutting them into rectangular patches. To embed superpixels, current encoders crop them to bounding boxes and then pad, resize, or mask them, so background or fill values can leak into the node features. This matters in practice, and especially in medical imaging, where a prediction should depend on the tissue and not on an arbitrary preprocessing choice: on BACH, changing only the fill of the masked region shifts the accuracy of frozen encoders by up to 6 percentage points, larger than the differences among the four best encoders under mean fill, and the preferred fill differs between datasets, so it cannot be fixed once and for all. We present TAN (Tissue-Aware Network), a compact convolutional encoder (0.41M parameters) that processes each superpixel at its native shape and extent and excludes background from every operation, which makes its embeddings exactly invariant to the fill by construction. Under identical masks and an identical graph classifier, TAN's accuracy on BACH is not significantly different from that of the 303M-parameter foundation model UNI (81.1% vs. 80.7-82.6%, depending on the fill), with ~740x fewer parameters, ~1,190x fewer FLOPs, and no pretraining. On BRACS, TAN is close to a ResNet18 trained under the same protocol and trails the histopathology foundation models. An ablation with TAN's architecture but standard processing of fill-padded crops shows that masking keeps accuracy similar (nominally higher on BACH, lower on BRACS, within noise) at less than half the FLOPs.