Shared-factor model helps predict image patches more accurately
Generative Residual Factorization
Computer Vision and Pattern Recognition
Summary
This paper looks at how to better predict the next part of an image using two ideas: a shared scene factor and a leftover part called the residual. The authors show how these parts can be separated and how past image information can improve understanding of the scene factor but not the residual. They also demonstrate this concept with a simple example using Gaussian models. Essentially, their work helps break down the uncertainty in predicting images into parts that can and cannot be improved by better past information.
What this means in practice
- •For computer vision engineers: Design improved models for image patch prediction by separating predictable scene factors from inherent randomness in image data.
- •For signal processing teams: Develop better algorithms for sequential data prediction by applying shared-factor and residual decomposition to reduce uncertainty.
A theory result. No direct application yet.
Authors
Letian Gong, Yuzhou Hong
Abstract
Under a shared-factor model, the conditional law of the next image patch factors into a posterior over the shared scene factor and a residual kernel given that factor. A sufficient statistic of the past replaces the raw past in the posterior and does not replace the kernel. The conditional entropy splits into residual entropy, which no observation of the factor can remove, and a posterior term, which a better representation of the past can remove. Next-embedding prediction is a directional likelihood on a shallow map, so the fiber of that map is unidentified and a constant embedding remains a minimizer. The same split is an equality in a scalar Gaussian model, evaluated in closed form.