Papers for

image generation developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Structural self-teaching improves image generation with complex prompts

Structural Self-Teaching for Compositional Generation in Unified Multimodal Models

Abstract: We propose Structural Self-Teaching (SST), a novel framework that enhances compositional image generation in diffusion-based unified multimodal models (UMMs) by leveraging their internal understanding path as a source of structural supervision. The generation path of UMMs often struggles with dense compositional prompts, leading to missing objects, attribute binding failures, and spatial errors. In contrast, their understanding path localizes the same phrases in images that satisfy these prompts. SST converts this phrase-level grounding into training-time structural signals for the generation path. Specifically, we extract phrase-conditioned support masks from internal attention maps derived from the understanding path and employ them through two mechanisms. Phrase-local contrastive alignment uses these masks as pooling regions and aligns each generated phrase region with the same region of the target image. A structural scaffold builds a phrase-level layout from the same masks and modulates an intermediate spatial state of the generation path during training. The scaffold is phased out during training and removed at inference. SST requires no external structural annotations for training and no additional input at test time. Extensive experiments across multiple UMMs demonstrate that SST improves compositional fidelity over LoRA fine-tuning on the same data, with no inference-time cost. Project page is available at https://bellos1203.github.io/sst

Fri 13 NovComputer Vision and Pattern Recognition
The gist
Generating images from detailed descriptions is hard for current AI models because they often miss objects or place things incorrectly. The authors propose a method called Structural Self-Teaching that uses the model's own ability to understand images to guide how it creates them. By teaching the generation part of the model where each described object should go, without needing extra labels, the images better match the complex prompts. This method works during training without slowing down image creation later.
Open → 2611.18261v1

Diffusion models localized memorized image regions without contrast maps

JaRS: Contrast-Free Localization of Memorized Regions in Diffusion Models via Jacobian Row Sums

Abstract: Diffusion models sometimes memorize and reproduce regions of their training images, raising privacy and copyright concerns. We aim to localize these regions. Existing contrast-based methods use the diagonal of the denoiser Jacobian, which reflects the posterior variance of each pixel. However, a small diagonal entry is ambiguous: it signals either memorization or averaging over many neighboring pixels in smooth or predictable regions. To resolve this ambiguity, these methods subtract a reference map, which requires a second evaluation with a null prompt or an earlier checkpoint. Unconditional models have no null prompt, and earlier checkpoints are unavailable in general. We show that Jacobian row sums (JaRS), unlike the diagonal, separate memorization from local averaging without a reference map. Pairing JaRS with an image-level detection score that uses the same statistic at an early denoising step yields JaRS+. Across Stable Diffusion v1.4 and v2.1, Realistic Vision v5.1, and an unconditional diffusion Transformer (DiT), JaRS+ outperforms contrast-based localization methods.

Fri 13 NovComputer Vision and Pattern RecognitionMachine Learning
The gist
Diffusion models sometimes remember and copy parts of images they were trained on, which can cause privacy or copyright issues. The authors developed a way to find these copied regions by analyzing how the model changes pixel values, using a new technique called Jacobian row sums (JaRS). Unlike previous methods, this doesn't need extra reference images or prompts, making it easier to apply across different types of models. Their improved method, JaRS+, works better for spotting memorized content in popular diffusion models.
Open → 2611.18024v1

Diffusion free score matching enables exact conditional sampling

Diffusion-free Score Matching for Exact Conditional Sampling

Abstract: The success of modern score-based generative models is largely driven by diffusion-based training, which learns a collection of time-indexed score fields corresponding to progressively noisier versions of the data distribution. While such models achieve impressive sample quality, leveraging them for sound probabilistic inference---particularly sampling from arbitrary conditionals of the model---remains challenging. This difficulty arises because conditioning information is only available for clean data and not for the higher noise levels required to generate exact conditional samples. In this paper, we introduce DIffusion-free SCOre matching (DISCO), a principled approach that sidesteps the need for time-dependent score fields altogether. Our method is based on an objective that, while reminiscent of diffusion-based training, estimates only a single score of the (slightly perturbed) data distribution, for which the exact conditional score is readily obtained. In experiments, DISCO is competitive with state-of-the-art diffusion models in terms of sample quality. More importantly, DISCO yields a more faithful representation of the underlying data distribution and---crucially---enables accurate sampling from arbitrary conditionals, outperforming standard heuristic samplers. This capability opens the door to principled and flexible probabilistic reasoning with score-based models.

Fri 6 NovMachine Learning
The gist
Generating realistic data, like images or sounds, often involves adding noise and learning to reverse this process. Traditional methods rely on multiple noisy steps, which makes it hard to generate data conditioned on specific features. The authors propose a new approach called DISCO that avoids multiple noise levels and learns a simpler, single-step model. This makes it easier and more accurate to generate data based on specific conditions, improving flexibility and reliability in data generation.
Open → 2611.09758v1

LongBanana benchmark measures image creation from many references

LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis

Abstract: We introduce LongBanana, to our knowledge the first expert-designed and expert-verified benchmark for long-context multi-reference image synthesis in the age of agent-driven image generation. The task requires retrieving dispersed visual evidence, rejecting distractors, binding entities and relations, and composing one coherent final image from multiple references. LongBanana contains 90 carefully designed tasks organized by a 3-by-3 taxonomy; each task includes about 32 reference images, 20 constraints, and 50 evaluation items, yielding 4,670 expert-specified atomic evaluation items in total. Its graph-grounded evaluation design instantiates each task with reference images and an entity-relation graph, then converts node, edge, and whole-graph obligations into checklist items for Content Fidelity, Relational Binding, and Global Constraint Satisfaction. Reviewed expert labels include a 15% double-labeled subset with 89.4% exact agreement and 0.83 quadratic weighted kappa. We further introduce LongBanana-Gen, an agent-harness framework for long-horizon image generation; experiments show that minimal agent harnesses improve over direct generation, while a ReAct-Plan-Exec-Reflect (RPR) workflow yields harness-dependent gains by strengthening evidence grounding and repair. Together, LongBanana provides an auditable testbed for measuring dense visual evidence use and a shared target for developing stronger agent-driven image-generation systems. Code and data are available at the project page: https://long-banana.com/.

Wed 4 NovComputer Vision and Pattern RecognitionComputation and Language
The gist
Creating one clear picture from many different images is very hard, because you have to pick the right parts, ignore distractions, and put everything together correctly. The authors made LongBanana, a new test with 90 special tasks to check how well computers do this. Each task gives many images and rules, and experts carefully marked exactly what the final picture should show. They also made a system to help computers improve by thinking and fixing mistakes step by step. This work helps make better tools for making pictures using lots of information.
Open → 2611.06856v1

SDMD improves few-step image generation by better matching data distributions

sDMD: Sinkhorn Distribution Matching Distillation

Abstract: Distribution Matching Distillation (DMD) distills multi-step diffusion models into few-step generators by minimizing reverse KL divergence. Despite its strong performance, DMD guides samples by local density alone, which introduces two limitations: teacher scores can be unreliable in regions of low teacher density, and the mode-seeking behavior of reverse KL can collapse the student onto a subset of the teacher modes. We first connect DMD to Wasserstein gradient flow, showing that its gradient follows from a fixed-point formulation with integral KL (IKL) as the energy functional. This perspective motivates augmenting IKL with an optimal transport term that addresses these limitations by guiding outlier samples toward the target and promoting mode coverage. We realize this term using Sinkhorn divergence estimated from finite batches. In practice, however, errors in the estimated Sinkhorn velocity can perturb in-domain samples for which IKL guidance is already accurate. To this end, we introduce sDMD, which adaptively activates the Sinkhorn correction based on discriminator outputs, focusing on outlier samples where IKL may be unreliable. With 2--4 sampling steps on FLUX.1-dev, Qwen-Image-20B, and Wan2.1-14B, sDMD outperforms distillation baselines on GenEval, DPG, WISE, and VBench while improving sample diversity over DMD2.

Thu 29 OctComputer Vision and Pattern RecognitionMachine Learning
The gist
Generating high-quality images quickly is difficult because detailed models usually require many slow steps. The researchers studied a method called distribution matching distillation (DMD), which trains faster models to mimic slower ones by focusing on local data details. They found that DMD sometimes misses important variety in images and struggles with rare cases. Their new method, sDMD, adds a way to gently guide samples towards the right areas without disrupting well-learned parts. This helps generate diverse and accurate images using fewer steps than before.
Open → 2610.39889v1

New method improves digital image coding with full code use

Transporting Quantiles to Codebook: Scalable Vector Quantization without Codebook Collapse

Abstract: Effective discrete representation learning with VQ-VAE serves as a fundamental component of modern autoregressive generative models. However, it is widely known to suffer from codebook collapse and inefficient code utilization. To address this issue, we propose FlowVQ, a simple yet effective alternative to standard vector quantization by formulating neural quantization as a joint modeling and clustering process. Specifically, we learn the latent space using a lightweight flow matching model alongside training, corresponding to a probability flow ODE that transports an often fixed well-behaved source distribution while preserving probability mass. Based on this, we define the codebook C_1 by transporting even-quantile partitioned prior codes C_0 from the source space through the learned flow, and obtaining the Voronoi centroids under the induced distribution, leading to a structured discretization. We evaluated on image tokenization and generation, as well as on two tasks requiring high-precision tokenization, demonstrating scalability and improved expressivity of the codebook with full utilization.

Mon 26 OctMachine Learning
The gist
Digital tools often compress images using a set of code pieces, but sometimes these codes aren’t used efficiently, limiting quality. The authors propose FlowVQ, which learns to group image data better by smoothly moving simple starting codes into a structured set that covers the data well. This approach avoids the common problem where some codes don’t get used and helps capture images more precisely. They tested it on image generation tasks and showed it scales well while improving how much of the code set is used.
Open → 2610.34828v1

Benchmark reveals gap between realistic images and exact identity matching

LEVI: Benchmarking Visual Identity Fidelity with Verifiable Low-Entropy Constructs

Abstract: Modern generative models can produce visually plausible images while violating exact identity constraints. Evaluating such failures requires distinguishing violations of the requested identity from permissible variation in appearance. We introduce Low-Entropy Visual Identity (LEVI), which characterizes visual identities determined by compact facts and explicit rules. The low-entropy requirement concerns identity uncertainty, while appearance may remain diverse. We formulate identity fidelity in terms of the probability that a conditional image distribution assigns to the identities permitted by the prompt. Visual Identity Spaces (VIS) formalize this distinction by specifying admissible identity values and which different realizations express the same identity. Building on this formulation, we develop Verifiable Visual Grammar (VVG), a declarative language whose typed primitives and compositional constraints describe identity-bearing structures. Using this grammar, we build a benchmark for image generation. It assesses each output against the task's identity requirements, drawing evidence from the final pixels. The curated benchmark contains 1,375 tasks spanning 120 base identity spaces across six domains. Among ten evaluated model configurations, the highest whole-task pass rate is 58.69%. These results reveal a persistent gap between visually plausible images and faithful realization of their specified identities. Project page (forthcoming): https://maxiao0234.github.io/LEVI/.

Mon 19 OctComputer Vision and Pattern Recognition
The gist
Generating images that look real is easier than making them match a specific identity exactly. The authors created a way to test if generated images faithfully follow the requested identity rules by using low-entropy identity concepts. They built a language and benchmark to check how well models stick to identity constraints, finding that even the best models fail to fully meet the requirements. This shows that creating images that both look good and are exactly right is still challenging.
Open → 2610.25716v1

Token merging improves speed and quality in image diffusion models

Which Tokens to Merge? Diffusion Dynamics for Efficient Image Generation

Abstract: Token merging has emerged as an effective training-free strategy for accelerating diffusion models by reducing redundant token computation while minimizing generation degradation. A key challenge is determining which tokens can be safely merged and which should be preserved for subsequent computation. Existing method addresses this by preserving prompt-aligned foreground regions, thereby protecting semantically relevant structures. However, this text-relevance criterion overlooks background objects, local details that are not explicitly mentioned in the prompt, leading to degraded structural coherence. In this work, we propose Dynamics-aware Token Merging (DaTMe), a training-free framework that grounds token merging in the underlying diffusion dynamics. Our key insight is that token importance should reflect whether a token remains unresolved during denoising and whether it carries structural information. To this end, DaTMe constructs a dynamics- aware importance map from the Tweedie denoised estimate. The map combines a temporal signal, measuring inter-step changes in the predicted clean image, with a structural signal, capturing local high-frequency content such as boundaries and fine details. Using this map, DaTMe adaptively assigns tokens into merging roles under a given compression ratio: important tokens are preserved as anchors, while tokens that are both temporally stabilized and structurally homogeneous are selected as safe merge candidates. Experiments on PixArt-a and FLUX demonstrate that DaTMe maintains visual fidelity across the entire image, highlighting diffusion dynamics as an effective token-level merging decision in efficient generation. Our code is available at https://github.com/SeungJuCha/DaTMe.

Thu 15 OctComputer Vision and Pattern Recognition
The gist
Generating images using diffusion models can be slow because many tiny pieces, called tokens, are processed. The paper introduces a way for the computer to decide which tokens are important to keep and which can be safely combined, based on how the image changes during the creation process and where detailed edges appear. This method helps speed up image generation while keeping pictures looking sharp and correct. The authors show their approach works well on different image sets, improving both speed and quality without extra training.
Open → 2610.22370v1

Prediction horizons affect image quality depending on timing and state

Rethinking Prediction Horizons: Decoupling Model Queries from State Updates in Few-Step MeanFlow

Abstract: MeanFlow predicts interval-average velocities, but conventional few-step sampling ties the interval queried from the model to the duration of the state update. We separate these two quantities at inference, fixing the model, update grid, and number of queries while varying the prediction horizon. Our central finding is that the horizon response depends on both query position and the states entering that query. Across seven independently trained CIFAR-10 checkpoints at classifier-free guidance (CFG) scale 2, a longer horizon worsens FID at early steps but improves it at the midpoint. To probe whether query time alone determines this response, we fix the midpoint query and current/subsequent guidance while changing only earlier CFG. In held-out factorial outcomes on five previously studied checkpoints, the mean midpoint horizon benefit increases from 0.109 to 1.308 FID, indicating sensitivity to the incoming-state distribution. A frozen four-query rule reduces mean CIFAR-10 FID by 2.023 without retraining or extra queries, including gains on five checkpoints trained after rule selection. A milder rule also improves mean FID by 1.041 on a stronger frozen grid, with fixed-rule FID gains on the evaluated public ImageNet-256 and AFHQ checkpoints. However, matched-budget calibration favors query-start shifting in historical CIFAR tests, and the tested Horizon allocation adds no FID benefit beyond Grid+Start on two public ImageNet models. Thus, prediction horizon is a distinct, position- and state-sensitive inference control, but not a universally preferred one.

Wed 14 OctMachine LearningArtificial Intelligence
The gist
Prediction horizon refers to how far ahead a model looks when making predictions, usually linked to time intervals. This paper shows that changing the prediction horizon separately from how often the model updates its state can impact the quality of generated images. The authors found that the effect depends not just on timing but also on the states the model receives at each step, and that adjusting prediction horizons can improve or worsen results depending on the setup. They also demonstrated rules that improve image quality without retraining the models by changing how queries to the model are scheduled during inference.
Open → 2610.20071v1

Training-free conditional sampling for pretrained Schrödinger bridge models

BridgeTwist: Twisting Schrödinger Bridges for Training-Free Conditional Sampling

Abstract: Generative models are often required to serve many downstream conditional tasks. In the training-free setting, a pretrained unconditional model is reused as a prior and combined with a task-specific likelihood to sample from the corresponding posterior. While such training-free samplers have been developed for diffusion and flow models, their counterpart for Schrödinger bridge (SB) models remains unexplored, despite the growing appeal of SBs as flexible stochastic transport models between general source and target distributions. We propose BridgeTwist, a training-free conditional sampler for pretrained SB generative models, formulated as a tempered twisted sequential Monte Carlo scheme. Its twist is a closed-form plug-in surrogate, obtained by combining the velocity and score fields exposed by SB pretraining into an endpoint predictor and evaluating the likelihood there. Since this surrogate is less reliable near the source, we temper the twist with a time-varying exponent, downweighting its influence at early steps. We further introduce a bridge-shaped inference-time diffusion schedule that concentrates the proposal noise in the interior of the path. Telescoping importance weights then cancel all intermediate twist factors, yielding an asymptotically exact sampler for the posterior. Empirically, BridgeTwist outperforms training-free baselines on class-conditional and text-prompted image generation, and is competitive with specialized inverse problem solvers on inpainting, deblurring, and super-resolution.

Tue 13 OctMachine Learning
The gist
Generating images or data based on certain conditions usually needs special training, but this work finds a way to do it without extra training. The authors created BridgeTwist, a method that uses a pretrained model to produce data matching a specific condition by adjusting how it generates samples. It balances where it trusts its guesses along the generation path and carefully combines information to make accurate outputs. BridgeTwist works well for tasks like creating images from text, filling in missing parts, or enhancing image quality.
Open → 2610.18658v1

Language based ordering improves image coding and captioning quality

Making Language the Low-Rate Code of Vision: Successive Refinement from Words to Pixels

Abstract: An ordered visual code lets a short prefix stand in for an image, and the training loss decides what that prefix contains. Under pixel reconstruction the first bits go to the directions of highest variance, and we prove that a caption can then be independent of the entire first half of a code that is rate-optimal for pixels at every length. In the rate-distortion sense a caption is therefore not the low-rate code of its image, unless the code is built to make it one. We study that construction and what it costs. We cast captioning and text-to-image generation as successive refinement under a distortion measure that changes along the code, from caption log-loss in the early stages to pixel distortion in the last, so that each stage is compared with its own optimum. Putting words first then costs nothing among the caption stages exactly when the captions form a summarization chain, and it costs the pixels at most the caption uncertainty left in a pixel-optimal reconstruction, which is large for short prefixes and small at full length. We build RUNG, an ordered code whose prefixes are supervised by a summarization ladder of captions and whose full length reconstructs the image; generation extends the code past the prompt and captioning truncates it. With data and capacity held fixed, ordering by language instead of reconstruction raises prefix-caption information 2.9 times and GenEval from 0.76 to 0.83, at a measured cost of 2.8 dB PSNR at a quarter of the code and 0.1 dB at full length. RUNG-XL (3.4B) reaches 0.88 GenEval and 87.6 DPG-Bench in 32 network evaluations and 0.91 GenEval when errors are redrawn at the rung where they appear; its coarse and fine captions agree 94.1% of the time, and it edits images without inversion.

Mon 12 OctArtificial Intelligence
The gist
Images can be compressed into codes that let you see a rough version first and finer details later. Usually, those codes focus on pixels and not on text descriptions (captions). The authors show that if you build the code to prioritize language early on, you get better captions without much loss in image quality. They created a system called RUNG that puts words first, then pixels, improving how well captions match images and how images are generated from text. This helps machines understand and create images more like humans do, by starting from language and refining details step by step.
Open → 2610.16552v1

MixTok combines global and local tokens to improve image encoding

MixTok: Adaptive Visual Tokenization with Global 1D Tokens and Sparse 2D Refinement

Abstract: Efficient visual tokenization requires balancing representation compactness with the preservation of fine-grained image content. Conventional grid-based tokenizers allocate tokens uniformly across spatial locations despite non-uniform visual complexity, whereas compact 1D tokenizers aggregate global information but can struggle to preserve local details. We introduce MixTok, a hybrid visual tokenizer that combines a compact 1D global representation with sparse 2D tokens for local refinement. A content-adaptive router ranks candidate locations by refinement priority and determines the number of 2D tokens for each image under a target average token budget. At the highest-scoring locations, 2D token features replace the spatial features recovered from the 1D representation, and a shared decoder reconstructs the image from the resulting mixed features. On ImageNet 256 x 256, MixTok achieves highly competitive reconstruction and generation performance, with an rFID of 1.195 and a gFID of 1.95 at respective average budgets of approximately 128 and 131 content tokens per image.

Fri 9 OctComputer Vision and Pattern Recognition
The gist
Encoding images efficiently means representing important details without using too much space. The authors introduce MixTok, a new method that uses both a global summary of the whole image and extra detailed tokens in selected areas with complex content. This approach adapts to each image, adding detail tokens only where needed while keeping the total number of tokens low. Tests show that MixTok can reconstruct and generate images with quality comparable to other methods, but using fewer tokens.
Open → 2610.13181v1

Quadtree image tokenizer improves visual generation efficiency and control

QuadTok: Quadtree Visual Tokenizer for Autoregressive Image Generation

Abstract: We introduce QuadTok, a novel framework for visual tokenization and autoregressive image generation. Compared to traditional approaches using 2D grids or 1D token sequences, we propose a hierarchical quadtree structure, bridging the gap between 2D spatial binding and 1D sequence-level flexibility. The QuadTok tokenizer dynamically allocates representational capacity to visually intricate areas while leaving homogeneous regions at a coarse resolution. Compared with a fixed 256-token grid, our ImageNet-trained tokenizer saves approximately 10% of tokens on ImageNet and 9% when transferred zero-shot to the COCO dataset, while maintaining comparable reconstruction fidelity. Furthermore, the natural causality introduced by the tree structure seamlessly enables autoregressive image generation. Conditioned on a quadtree topology supplied before generation, our 947M GPT-style generative model achieves a 2.08 gFID on the ImageNet $256 \times 256$ benchmark. Additionally, leveraging the strong spatial correlation preserved by the quadtree structure, the QuadTok generator enables zero-shot spatially controlled image generation capabilities. Code: https://github.com/myc634/QuadTok.

Wed 7 OctComputer Vision and Pattern Recognition
The gist
Images can be broken down into smaller parts for a computer to understand and recreate them. The authors created a new way to split images using a quadtree, which focuses more tokens on detailed areas and fewer on simple areas. This method saves computing resources while keeping the image quality high. Their approach also helps generate images step-by-step and allows control over where details appear in the generated image.
Open → 2610.10497v1

Energy-aware noise improves image diffusion model quality

Energy-Conditioned Noise Schedule and Whitening for Spectral Diffusion

Abstract: This paper introduces an energy-adaptive noise scheduling and whitening strategy for transform-domain diffusion models. Existing spectral diffusion methods account for the non-uniform statistics of transform coefficients through coefficient scaling, normalization, or frequency prioritization, while the forward diffusion noise schedule remains largely independent of the underlying spectral-energy distribution. We investigate whether the temporal evolution of the forward diffusion process should also follow the spectral organization of natural images. The proposed formulation combines global spectral whitening with energy-conditioned noise allocation that jointly modulates the injected noise according to the energy of individual transform coefficients and an image-dependent energy path over diffusion time. The resulting forward process preserves Gaussian transitions with closed-form marginals and remains compatible with standard DDPM and DDIM procedures without modifying the diffusion architecture. Experiments on CIFAR-10 demonstrate the contribution of the proposed energy-conditioned noise schedule and spectral whitening, reducing Fréchet Inception Distance from 142.48 for a compact DCTdiff U-Net variant to 100.45.

Mon 5 OctArtificial IntelligenceComputer Vision and Pattern RecognitionEmerging Technologies
The gist
Diffusion models generate images by gradually adding and removing noise. Traditional methods add noise uniformly without considering the energy of different parts in the image's frequency components. The authors introduce a way to adapt how much noise is added based on the energy of each frequency component, making the process more natural and effective. This approach improves image generation quality without changing the core model design.
Open → 2610.07206v1

Loop flow transformers improve image generation with less computation

LiFT: Loop Flow Transformers

Abstract: We introduce Loop Flow Transformers (LiFT), a family of looped generative models that scales computation by repeatedly applying a shared Diffusion Transformer (DiT) core, with only light changes to the standard architecture. Rather than asking every recurrent step for the final prediction, LiFT trains each step with a single regression target: a point on a straight path from the model's initial estimate to the flow-matching target. Because we index these targets by a continuous depth coordinate, a trained model can loop far beyond its training depth with no retraining, early exits, or other modifications. In our experiments, these longer rollouts improve generation, so inference computation can grow without adding parameters. On ImageNet at 256x256, LiFT-L/2 achieves an FID 3.34 points lower than our dense DiT-XL/2 baseline while using approximately 60% fewer parameters, 32% fewer training FLOPs, and 52% fewer inference FLOPs.

Sun 4 OctMachine LearningArtificial Intelligence
The gist
Generating high-quality images with AI often requires a lot of computing power. The authors present a new model called Loop Flow Transformers that repeatedly uses the same core process multiple times during generation. Instead of predicting the final image all at once, their method gradually improves the image over several steps, each trained to move closer to the target. This approach allows the model to generate better images with fewer parameters and less computation.
Open → 2610.05538v1

Best-of-N guidance improves alignment of diffusion model outputs

Best-of-$N$ Guidance for Test-time Diffusion Alignment

Abstract: Diffusion models achieve strong generative performance but often struggle to align generated samples with human preferences measured by a reward model. A simple yet effective algorithm for test-time alignment is Best-of-$N$ (BoN) sampling, which draws $N$ i.i.d. samples from a pre-trained diffusion model and outputs the single highest-reward sample. Despite its empirical success, BoN makes limited use of reward information, as it is incorporated only at the final selection stage without influencing the reverse diffusion trajectory during sampling. Consequently, BoN sampling does not improve the average alignment of generated samples and is primarily suited to single-output settings. We propose Best-of-$N$ Guidance (BoNG), a novel method that integrates the principle of BoN sampling directly into the reverse diffusion process. BoNG performs online BoN selection over denoising particles and adjusts the reverse diffusion process to steer the particle population toward higher-reward regions during generation. Specifically, by introducing an asymmetric guidance interaction among denoising particles, BoNG uses the current BoN particle as a guidance signal to the rest of the particle population. This particle-level interaction reshapes the sampling process toward higher-reward regions, enabling BoNG to improve not only the final best sample beyond Vanilla BoN sampling, but also the average quality of generated samples. Over 36 empirical comparisons, BoNG achieves the best performance in 29 cases, ranking first in 80.56% of the comparisons against SMC and Vanilla BoN sampling. BoNG also supports multi-output capability, achieving 1.3$\times$ ImageReward score of the latest sample-based guidance method with a 1.6$\times$ speedup. We release the code at https://github.com/aailab-kaist/BoNG.

Sun 4 OctMachine LearningArtificial Intelligence
The gist
Diffusion models are AI tools that create images but sometimes miss what people really want. The authors found that the usual way of picking the best generated image doesn’t improve the overall quality or how well the images match human preferences. They propose a new method called Best-of-N Guidance (BoNG) that adjusts the image creation process itself to favor better images earlier on. This method makes the whole set of generated images better, not just the best one, and runs faster than other advanced methods.
Open → 2610.05108v1

Portrait aesthetic scoring improves by matching photographic intent

PortraitAes: Intent-Conditioned Structured Portrait Aesthetics Assessment

Abstract: Portrait aesthetic assessment assigns comparable scores according to how effectively human-centered images fulfill their photographic intent. These scores support data filtering, candidate selection, and preference modeling in image-generation pipelines. Existing methods typically predict a single aesthetic score or use general-purpose MLLMs without conditioning on photographic intent. This omission matters because the same blur, pose, lighting, or framing choice may serve one photographic intent but undermine another. These models thus learn context-agnostic aesthetic priors and yield inconsistent, inaccurate, misleading judgments for portraits with distinct photographic objectives. We introduce PortraitAes-Bench, an 11K-scale benchmark that decomposes this task into intent-conditioned subjudgments. Expert-authored rubrics define nine photographic intents, six first-level dimensions, and 22 secondary criteria. They support a structured pipeline for intent routing, specialist assessment, verification, and score fusion. Following this structure, we train PortraitAes with multi-task supervision. We then improve score comparability through Gaussian score calibration and within-dimension cross-image ranking. On the standard benchmark, PortraitAes achieves a Pearson correlation of 0.924 and a Spearman rank correlation of 0.934. On the hard-case set, its Pearson correlation is 0.829 and its Spearman rank correlation is 0.795. Across both sets, PortraitAes outperforms the evaluated general-purpose MLLMs and specialized aesthetic baselines.

Sun 4 OctComputer Vision and Pattern Recognition
The gist
The quality of a portrait photo depends on the photographer's purpose, like highlighting mood or style. Existing AI models rate photos without knowing this purpose, so they can make mistakes. The authors created a new dataset and system that considers nine different photographic goals and detailed criteria to score portraits more accurately. Their system outperforms others at judging portraits based on what the photo is trying to achieve.
Open → 2610.05010v1

Spectral correction improves guided image generation quality and consistency

Correcting Guided Diffusion Trajectories with Spectral Alignment

Abstract: The practical success of conditional image generation hinges on fine-grained differences in condition alignment and visual fidelity. Classifier-free guidance (CFG) is central to this success, but its lack of an explicit criterion makes it difficult to assess whether the guided trajectory is progressing as intended. To address this gap, we show that spectral alignment provides a principled criterion for understanding guidance behavior and improving guided diffusion sampling through adaptive correction. Our analysis identifies the spectra of intermediate states as an indicator of consistency with the expected spectral evolution of the forward process. Based on this observation, we introduce Spectral Correction Guidance, a method that corrects deviations from an analytic reference spectrum during sampling. The proposed method is training-free and applicable across diffusion backbones and conditional generation tasks without modifying the underlying model. Experiments demonstrate consistent gains in preference-based metrics over baseline guidance methods in text-to-image generation and improved generation quality over CFG on ImageNet. These improvements persist across a range of guidance scales and with fewer denoising steps. Our analyses and ablations provide insight into guidance behavior and how the proposed method affects generation quality.

Fri 2 OctComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Generating images from text or other conditions often uses a technique called guided diffusion, which can be hard to know if it's working properly. The authors show that looking at the 'spectra'—patterns in the evolving image states—can reveal if the process is on track. They introduce a way to adjust the image creation steps based on these spectral checks, improving the quality without extra training or changing the base models. Their method works across different tasks and settings, making image generation more reliable and visually better.
Open → 2610.02753v1

Salient factor discovery improves subtype recognition and generation accuracy

SAGE: Salient Factor Discovery and Generation with Visual Foundation Representations

Abstract: Given a target dataset, such as faces with eyeglasses, and a background dataset, such as faces without, contrastive analysis separates \textit{salient} factors specific to the target from \textit{common} content shared by both. We aim for salient representations that capture target-specific detail in each image, such as the shape, color, and position of the glasses, so that they reveal subtypes without subtype labels and guide the generation of new examples of a discovered subtype, even one with no name or text description. We introduce SAGE, which learns both factors directly in the high-dimensional spatial latent of a frozen representation autoencoder and conditions a diffusion transformer on the learned salient representation of a reference image. On Digits-ImageNet and FFHQ eyeglasses, SAGE combines high-fidelity \textit{reconstruction} (rFID below $2$) with unsupervised \textit{subtype discovery}, recovering the digits better than baselines (probe accuracy $0.950$ vs.\ at most $0.281$) and revealing eyewear types, finer sunglasses styles, and mislabeled images; salient-conditioned \textit{generation} raises Digits-ImageNet subtype accuracy over the unfactorized latent ($90.5\%$ vs.\ $27.7\%$) and diversity on both datasets. On retinal OCT, SAGE's salient space separates three diseases using only normal/disease labels.

Wed 30 SeptComputer Vision and Pattern RecognitionMachine Learning
The gist
When given two groups of images, like faces with and without glasses, the authors’ method finds specific differences that stand out only in the group with glasses. This way, the method learns details about types of glasses without needing labels for those types. It can then generate new images showing these different glasses styles. Their approach also works for medical images, distinguishing diseases using only basic labels.
Open → 2609.39635v1

ExploreNet improves image generator exploration with adaptive noise

ExploreNet: Learning Where to Explore in Diffusion GRPO

Abstract: Group-relative RL methods such as Flow-GRPO post-train image generators by exploring with isotropic Gaussian noise added at every denoising step. This noise decides which rollouts the model learns from, yet it perturbs every channel and spatial position of the latent equally. In this paper, we instead show that latent elements differ in how much they change the generated image, so exploration should adapt to these differences. We introduce EXPLORENET to learn an adaptive exploration distribution. EXPLORENET is a policy that predicts a noise scale for every latent element from the current latent, the denoising step, and the prompt, before any reward is observed; it is trained on the reward spread of each rollout group and discarded after training, leaving inference unchanged. On Stable Diffusion 3.5 Medium, EXPLORENET improves held-out GenEval2 by 14% over Flow-GRPO, transfers to two independent compositional benchmarks and five preference and image-quality models, and reaches a 67.2% human preference win-rate. Overall, across our group-relative diffusion RL experiments, we find that exploration is learnable, the shape of the exploration distribution outweighs its magnitude, and rollout quality is more effective than rollout quantity.

Tue 29 SeptComputer Vision and Pattern Recognition
The gist
Image generators improve by learning from many attempts that start with some randomness. Current methods add the same random noise equally everywhere, but the authors show that different parts of the image’s hidden representation affect the final image differently. They created ExploreNet, which learns how much noise to add to each part based on the current state, making exploration smarter. This method leads to better image quality and higher human preference ratings.
Open → 2609.38329v1

Verifiable visual rewards improve image generation accuracy and generalization

Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts

Abstract: Precise instruction following in image generation, such as satisfying object counts and spatial relations, remains an open challenge at least in part because it is learned using unreliable reward models such as object detectors and vision-language models. We introduce Verifiable Visual Rewards (VVR), the first framework for programmatically verifiable image rewards, and show that training on it generalizes to natural prompts. Each VVR task is a scene of geometric objects and relations among them, from which we derive both the prompt and a deterministic verifier, so tasks can be generated in any number and at any chosen complexity. We release VVRBench, with 10,000 tasks over 32 constraint types, and VVRBench-Challenge, with 720 more complex tasks; the strongest model we evaluate---GPT-Image-2.5---solves 21.4% of VVRBench-Challenge. Using VVR scores as rewards for reinforcement learning (RLVVR) raises the accuracy of Stable Diffusion 3.5 Medium on VVRBench from 2.8% to 28.3% and demonstrates consistent easy-to-hard generalization. These gains extend to out-of-domain benchmarks, and mixing VVR into existing objectives further improves overall performance and human preference, motivating the adoption of VVR into standard image generation post-training recipes.

Mon 28 SeptArtificial IntelligenceComputer Vision and Pattern Recognition
The gist
Following detailed instructions in generating images, such as getting the number and placement of objects right, is hard because current ways to judge success are unreliable. The authors created a new approach called Verifiable Visual Rewards (VVR), which uses simple geometric scenes where they know exactly what the correct image looks like. They made a large set of tasks and showed that training models with these clear rewards helps models follow complex instructions better, even on real-world prompts. This method improves the quality and consistency of generated images as measured by both automated tests and human preferences.
Open → 2609.35641v1

Unbalanced optimal transport improves one-step image generation quality

One-Step Generative Modeling via Unbalanced Optimal Transport

Abstract: Drifting models enable one-step generation by amortizing distribution transport into training, but this efficiency places greater demands on the transport field estimated at each update. In large-scale training, the field is computed from finite mini-batches of generated and real samples, which provide only imperfect approximations to the underlying distributions. Balanced optimal transport enforces exact mass matching within every mini-batch, making the estimated field sensitive to the particular composition of the real-data batch. We find that generated and real samples should be treated asymmetrically: letting the mass assigned to real samples adapt while keeping every generated sample fully transported improves generation across six feature-space metrics in controlled ablations, and is more robust to the relaxation strength than relaxing both marginals simultaneously, which falls below balanced transport under stronger relaxation. Motivated by this observation, we propose Unbalanced Optimal Transport Gradient Flow (UOT-GF), which keeps the generated-sample marginal fixed and relaxes only the real-data marginal. Under identical settings at DiT-B/2 on ImageNet-256, UOT-GF improves Fréchet Inception Distance (FID) from 1.53 to 1.46 over the balanced W-Flow baseline; scaling the same recipe yields 1.34 and 1.22 FID at L/2 and XL/2, the best FID among the one-step models we compare. We further derive the induced UOT transport force, establish a kinetic Vlasov--Fokker--Planck formulation whose overdamped zero-temperature limit recovers the drifting dynamics, and characterize non-target stationary states together with sufficient conditions for convergence.

Sat 26 SeptInformation TheoryMachine Learning
The gist
Generating images in a single step is faster but can be tricky because the model tries to match two sets of images exactly, which can cause mistakes during training. The authors found that allowing the real images’ importance to change while keeping generated images fully accounted for leads to better picture creation. They designed a method called Unbalanced Optimal Transport Gradient Flow (UOT-GF) that uses this idea and showed it improves image quality on ImageNet. They also studied the math behind this method to understand why it works and when it succeeds.
Open → 2609.32708v1

Adaptive joint attention speeds up conditional image generation with diffusion transformers

RefAdapt-DiT: Adaptive Joint Attention for Reference-Conditioned Diffusion Transformers

Abstract: Diffusion Transformers (DiTs) have become the standard backbone for high-quality generative modeling, yet deploying them in conditional generation tasks remains computationally prohibitive because bidirectional joint attention repeatedly processes large reference streams. While existing optimization schemes mitigate generic temporal redundancy, they typically rely on coarse-grained static reuse and overlook the distinct dynamics of references and targets. Specifically, we observe that reference representations often evolve slowly along the generation trajectory, while the target often assigns little attention mass to them; reference drift and this target-to-reference exposure jointly shape how strongly stale reference states affect the target. To exploit these patterns, we introduce \RefAdapt, a training-free framework for adaptive control of joint attention between references and targets. Instead of rigid static strategies, \RefAdapt combines consecutive target-Q change with previously observed target-to-reference attention mass to control reference computation adaptively at block granularity. Under ultra-few-step settings, \RefAdapt enables speedups of up to $2.097\times$ on 4-step MiniMax H3 and $3.54\times$ on 8-step Qwen Image Edit, while maintaining comparable visual quality.

Sat 26 SeptComputer Vision and Pattern Recognition
The gist
Generating images with special AI models called Diffusion Transformers is usually slow when these models need extra information to create images, because they process reference details many times. The authors found that parts of this reference information often change slowly and are sometimes ignored by the model, meaning some computations are unnecessary. They designed a new method called RefAdapt that smartly decides when to update this reference information based on how much the model is paying attention to it, without retraining the AI. This approach speeds up the image generation process significantly while keeping similar image quality.
Open → 2609.32415v1

Compute allocation improves training quality in drifting image models

Refresh or Realize? Compute Allocation in Drifting Models

Abstract: Drifting Models train a one-step generator by recomputing a finite-sample drift field at every iteration and taking an optimizer step toward the drifted target. The field says how generated samples should move, but the step is taken in parameters shared by all samples, so the motion the network actually makes need not match the motion it was given. This leaves a basic training question open: should extra compute go into fitting the current target more closely, or into recomputing the field? We study it on ImageNet 256x256. Holding the target fixed for k optimizer steps and measuring the realized displacement, we find that deeper fitting does bring the network closer to the frozen target, and that the number of steps needed before it makes any net progress drops from about sixteen early in training to one later on. When the extra steps come for free, k=2 also lowers FID. Once they are paid for, the result flips: at approximately matched measured wall-clock, spending the budget on fresh fields gives lower FID than deeper fitting, on both training seeds. The target itself shows why a fresh field is worth so much. Redrawing the finite support rotates its direction far more than a parameter update does (cosine ~0.3-0.6 against ~0.95), and a correction that is optimal in field space is not reliably better in FID than a parameter-free one. For Drifting, fitting each target well and spending compute well are different goals.

Sat 26 SeptMachine Learning
The gist
This paper looks at how to best use computing power when training drifting models, a type of image generator. The models learn by moving images step-by-step based on calculated directions, but the directions change over time, so the model must decide whether to spend more time refining current directions or recalculating new ones. The authors found that early on, more focus on refining helps, but later, recalculating directions more often leads to better image quality. This insight helps balance computation during training to get better final images.
Open → 2609.32298v1

New multi-agent system scores image quality more reliably on open-ended traits

An Evolutionary Agentic Approach for Open-ended Image Quality Perception

Abstract: Generative models are rapidly expanding image quality assessment (IQA) beyond traditional fidelity factors to emerging dimensions such as physical plausibility and text-rendering correctness. However, existing IQA models rely on fixed definitions and heavy supervision, making them difficult to extend to open-ended perceptual dimensions. We identify holistic bias as an important limitation: when scoring an unseen dimension, models reuse generic quality priors, leading to scoring errors and rank inversion. To address this, we propose PACE (Perceptual Agentic Collaborative Evolution), a training-free multi-agent framework that formulates open-ended IQA as explicit protocol construction. Given a target dimension, PACE uses collaborative agents to construct an evaluation protocol composed of verifiable Visual Question Answering (VQA) probes, grounding evaluation in concrete visual evidence rather than holistic impressions. The resulting protocol is calibrated using only four human-annotated images per dimension, while a dual-track scoring mechanism aligns model perception with human scoring scales. Across traditional IQA, structural fidelity, context-aware aesthetics, and newly defined open-ended dimensions, PACE consistently improves its MLLM backbone, achieving competitive performance across diverse IQA settings, and reduces the Holistic Override Rate (HOR) from 44.4\% to 8.6\%.

Sat 19 SeptComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Assessing the quality of images involves more than just measuring sharpness or fidelity—it includes things like how natural or correct text in images looks. Existing methods struggle when asked to judge new kinds of quality because they rely on fixed ideas and lots of training. The authors developed PACE, a method where multiple AI agents work together to create clear, question-based tests instead of relying on broad impressions. This leads to better alignment with how people judge image quality on many different traits, even ones not seen before.
Open → 2609.22942v1

Hardware accelerator speeds up diffusion transformer machine learning models

The World Model Hardware Accelerator

Abstract: Diffusion transformers invert the arithmetic that autoregressive decoding made familiar. There is no token-by-token recurrence: every denoising step is a full-sequence forward pass over static shapes, so the entire schedule is known at compile time and the only serial dimension is the step count itself. We exploit that structure in WMHA, a latency-first diffusion-transformer inference accelerator: a very-long-instruction-word sequencer issues four engines from one instruction word, a weight-stationary 16x16 dual-dot array streams FP8 and BF16 contractions, and a single-pass online-softmax attention pipeline keeps keys and values resident through a skewed software pipeline. The design is specified in a frozen micro-architecture document, implemented in synthesizable SystemVerilog, and verified against a double-precision reference model by a UVM environment whose acceptance criterion is semantic: the device must run a real denoising trajectory and reduce mean squared error against a clean latent by at least a factor of ten. It does so by a factor of 23, at both synthesized configurations, with zero element failures across 237 million checked values. Eleven application benchmarks built from published model shapes, including the original diffusion-transformer configuration, run on the device and report measured occupancy beside separately labelled projections. Five engines are taken to routed layout in sky130 with parasitic-annotated timing and measured-activity power; the full chip is synthesized, and the host limit that stopped its place-and-route is quantified together with the machine that would remove it.

Mon 14 SeptHardware Architecture
The gist
Many machine learning models generate data step by step, which can be slow. The authors worked on a type of model called diffusion transformers that process all steps at once instead of one by one. They designed a special computer chip that speeds up these models by making the process more parallel and efficient. The chip was tested carefully and showed great accuracy and performance improvements for various model tasks.
Open → 2609.16244v1