Video tone mapping improved by learning to infer hidden brightness levels
Reframing Video Inverse Tone Mapping as Blind Posterior Inference via Latent Luminance Supervision
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Many videos are made for screens with less brightness range, but new displays can show much brighter colors and details. Changing a regular video to look good on a bright screen is tricky because we don’t know exactly how the video’s brightness was originally changed and some details are lost. The authors created a smart way to guess the original brightness using patterns inside the video and training the system on many examples with different brightness changes. This approach makes the bright parts of videos look more natural and accurate across different methods than before.
What this means in practice
- •For video editors: Produce HDR videos from SDR source footage that have more accurate brightness in clipped areas across varied tone mappings.
- •For video game developers: Improve generated game video content to look consistent in brightness when displayed on HDR-capable displays.
Authors
Hanwool Sul, Chang-Hoon Jeong, Kisung Shin, Bomseok Park, Donghoon Cha, Hansl von Kwon, DoHyung Ku, Byoung-Tak Zhang
Abstract
High-dynamic-range (HDR) displays are now common, but most video content, such as archival footage, broadcast material, and the output of generative video models, exists only in standard dynamic range (SDR). The need is sharpest for generated video, which has no HDR master at all and is graded alongside live-action and visual-effects material finished in HDR. Converting SDR to HDR means undoing a tone-mapping operation that was never recorded, and clipping destroys the radiance wherever it acts, so HDR recovery is a blind inverse problem. Methods built on generative video priors fill clipped regions with plausible structure, but their reconstructions are systematically biased: in every public method we test, saturated content leans too bright or too dark, and the direction of the lean moves with the tone operator behind the input. We call this failure the luminance error, measure both its magnitude and its sign, and make it the focus of this work. Reframing ITM as posterior inference under an unknown operator locates the failure in two places and yields two corrections. A fixed affine functional of the video-diffusion latent reads log-luminance without decoding; on that coordinate an asymmetric loss selects an expectile of the posterior, applied only where clipping leaves luminance unconstrained. We also synthesize training SDR through an explicit family of tone operators, so the model marginalizes over what the observation cannot show. Trained this way, our luminance error stays flatter across operators and smaller on average than any public method, and the full system achieves the best perceptual quality on all three benchmarks.