Light field primitives enable compact and fast novel view synthesis
Light Field Primitive for Novel View Synthesis
Computer Vision and Pattern Recognition
Summary
Creating new images of a scene from different angles can be slow and require lots of data. The authors introduce Light Field Primitives, a method that groups many rays of light into compact units to speed up rendering. This approach can quickly create realistic views with fewer resources and supports advanced effects like focus changes and transparent objects. It works by organizing these units in a way that matches how light travels, making rendering more efficient and flexible.
What this means in practice
- •For computer graphics developers: Create faster and more flexible rendering engines supporting advanced optical effects using fewer light field data points.
- •For augmented reality device makers: Implement efficient real-time novel view synthesis for AR headsets to render realistic scenes with refocusing and transparency.$Commercial implications: This method enables lightweight rendering engines that could power AR devices with rich visual effects, improving user experience and reducing hardware demands.
Authors
Liang Chen, Jiahui Ning, Xun Jiang, Xing Xu, Jimmy Ren, Fenglei Fan, Heng Tao Shen
Abstract
We present Light Field Primitives (LFP), a formulation for novel view synthesis that replaces the dense ray database with a compact set of differentiable primitives in the classical two-plane parameterization. Each primitive condenses a group of rays into one learned record, and its response to a query is governed by how closely that query belongs to the group. Rendering a camera ray then reduces to compositing all responses it elicits, and a scene can be optimized directly from posed images and rendered in real time with rays. Beyond its competitive performance on standard benchmarks, the main advantage of LFP is structural: its primitives reside directly in the 4D ray space, so optical and appearance effects that are already operations on the light field become behaviors of a single shared renderer. With minimal changes to that renderer, LFP supports multi-scale anti-aliasing, defocus deblurring with refocusing, rendering for fisheye cameras, and even transparent object reconstruction with ray refraction, matching specialized frameworks that devote substantial machinery to these effects.