When Geometry Aligns: Dihedral Hidden-State Transformations in UNet, ViT, and DiT Architectures
2026-07-03 • Machine Learning
Machine LearningComputer Vision and Pattern Recognition
AI summaryⓘ
The authors looked at how changing the internal parts of image-generating AI models in ways that keep or break their geometric patterns affects the models' stability and output quality. They tested these changes on different models, including Stable Diffusion's U-Net and transformer-based designs, measuring how these changes impacted the features inside the model and the final images. They found that making changes that respect the model's geometry helps keep the model stable, while inconsistent changes cause specific problems. Their work highlights that preserving geometric consistency is important when tweaking these kinds of AI vision models.
Diffusion modelsU-NetTransformersDihedral groupGeometric transformationsStable DiffusionSelf-Consistency ShiftFIDCLIP scoreLPIPS
Authors
Mojtaba Faramarzi, Alex Lamb, Irina Rish
Abstract
Diffusion architectures now encompass convolutional UNets as well as transformer-based designs such as Diffusion Transformers (DiTs), inspired by Vision Transformers (ViTs), yet the effects of structured geometric perturbations within these architectures remain poorly understood. We study this question through a unified framework that applies reflection-based elements of the dihedral group to intermediate hidden states as controlled internal interventions, contrasting geometrically consistent and inconsistent variants. Using activation-level diagnostics, including Self-Consistency Shift (SCS), Activation Mass Scatter (AMS), and Drift, we analyze feature stability and geometric drift. We find that consistent transformations improve stability, while inconsistent ones induce predictable, architecture-specific failures. In the main Stable Diffusion 2.1 U-Net study, we evaluate seven intervention modes over three seeds and complement the internal diagnostics with image-level FID, KID, CLIP score, and LPIPS diversity. Taken together with supporting ViT and controlled DiT analyses, these results establish geometric consistency as a key principle for stable hidden-state interventions in spatially structured vision and diffusion models.