AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)
2026-08-13 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors improved AlayaWorld by changing how the model uses conditioning signals, which guide what it generates. Instead of the old method, they now use a 3D point cache renderer and encode visual information in a way that matches the video’s timing and content more closely. They also made six specific changes to help the model better understand motion and memory over time. These updates help the system produce video content that is more consistent and better controlled by the input signals.
AlayaWorldconditioning signalschunk-wise autoregressive generation3D point-cache renderercausal-VAE latent spacemotion-aware latent conditioningspatial memorymemory dropoutAdaLNviewpoint control
Authors
AlayaWorld Team, Kaipeng Zhang, Chuanhao Li, Yifan Zhan, Yongtao Ge, Yuanyang Yin, Jiaming Tan, Kang He, Liaoyuan Fan, Mingliang Zhai, Ruicong Liu, Xiaojie Xu, Xuangeng Chu, Zhen Li, Zhengyuan Lin, Zhixiang Wang, Zian Meng, Zihui Gao
Abstract
This report presents an improved version of AlayaWorld. While the backbone architecture, chunk-wise autoregressive generation scheme, and training data remain unchanged from the previous release, we substantially revise how conditioning signals are represented and integrated into the model. The new design is guided by a simple principle: conditioning signals should match the generated content as closely as possible in both latent representation and temporal structure. To this end, we make two major changes. First, we replace the previous depth-warping-based spatial memory with a streaming 3D point-cache renderer. Second, we redesign the conditioning pipeline so that visual conditions are encoded in the same causal-VAE latent space, with temporal statistics consistent with those of the generated video. Concretely, the new version introduces six modifications: (1) replacing static-frame image conditioning with motion-aware latent conditioning; (2) causally encoding re-rendered spatial memory as a continuous sequence; (3) aligning the temporal-memory window in pixel space; (4) adopting hard memory dropout that removes memory tokens rather than zeroing them; (5) unifying the VAE encoding and decoding protocol across training and inference; and (6) removing the camera AdaLN branch, such that viewpoint control is provided entirely through the re-rendered spatial condition.