Papers for

video game designers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

GlyphAtlas improves text look in complex small or dense scenes

GlyphAtlas: Disentangling Stroke Detail from Layout for Scene Text Synthesis

Abstract: Scene text synthesis requires stroke-accurate characters rendered in a specified layout, as well as visual coherence with the surrounding scene. To achieve this, existing methods commonly pre-render the target text at its intended locations to form a glyph image that jointly specifies layout and glyph structure. However, this representation ties the resolution of each glyph to its target region, limiting fine-grained stroke guidance for small, densely packed, or structurally complex text. To tackle this, we propose GlyphAtlas, a scene text synthesis framework that retains the glyph image as a spatial anchor and supplies fine-grained stroke references through a dedicated glyph atlas. The atlas renders and encodes each character or word in a separate cell, decoupling reference resolution from target-region size. We further introduce atlas-anchored attention, which lets each text region retrieve stroke detail only from its corresponding atlas cells to refine the coarse glyph structure provided by the glyph image. This coarse-to-fine conditioning simplifies stroke rendering while preserving explicit layout correspondence. We optimize GlyphAtlas with a mask-weighted flow matching objective and a timestep-weighted text perceptual loss for writing accuracy and background consistency. Random resolution augmentation of atlas cells supports adjustable atlas budgets at inference. Extensive experiments demonstrate that GlyphAtlas achieves stroke-accurate synthesis of small, dense, and structurally complex text, zero-shot transfer to languages unseen during task-specific training, and leading performance in scenes with many text regions. project page: https://fr0zencrane.github.io/project-glyphatlas/

Sun 15 NovComputer Vision and Pattern Recognition
The gist
Creating realistic text in pictures needs letters that look exactly right and fit well into the scene. Previous methods combined letter shape and position in one image, which made it hard to show tiny details for small or crowded text. The authors came up with GlyphAtlas, which separates fine details of letters from their location, using a special map of characters. This helps produce clearer and more accurate text in images, even for small, busy, or new languages, and works well when many text parts appear together.
Open → 2611.20192v1

Visual reasoning guides efficient 3D object localization without training

QVR3D: Query-Guided Visual Reasoning for Training-Free 3D Grounding

Abstract: 3D visual grounding localizes objects in 3D scenes from natural language descriptions, requiring reasoning over semantics, spatial relationships, and multi-view observations. Training-free approaches avoid costly annotations but often rely on explicit scene transformation or scene-level frame selection, which may waste limited visual capacity on query-irrelevant regions. We observe that effective grounding should prioritize observations that provide evidence for the query rather than uniformly represent the entire scene. Based on this insight, we introduce QVR3D, a query-guided visual reasoning framework that operates on RGB-D observations using off-the-shelf foundation models. QVR3D combines Query-Guided Maximum Coverage (QGMC) for selecting complementary query-relevant views with Grounding Probability Estimation (GPE) that integrates instance confidence, attribute consistency, and relationship consistency. The framework supports both single- and multi-object grounding without task-specific adaptation. Experiments on ScanRefer and Nr3D achieve state-of-the-art zero-shot performance, while QGMC also improves frame efficiency across multiple 3D vision-language tasks. Our code is available at https://github.com/jwliao1209/QVR3D.

Sat 31 OctComputer Vision and Pattern Recognition
The gist
Finding objects in 3D scenes using natural language descriptions is hard because it requires understanding the scene deeply from multiple views. Liao and colleagues show that instead of looking everywhere equally, focusing on areas related to the description helps locate objects better without needing costly training. Their system, QVR3D, smartly picks the best views and estimates how well objects match descriptions using confidence and consistency checks. This approach works well even when locating multiple objects and sets new standards for zero-shot 3D grounding.
Open → 2611.01084v1

X-World 1 6 provides controllable virtual cities for driving tests

X-World 1.6: Building Controllable Virtual Cities with World Models

Abstract: Deploying generative world models for end-to-end closed-loop evaluation and policy learning requires precise scenario control, consistent observations, and sustained generation quality during extended interactions. This report presents \emph{X-World~1.6}, advancing X-World~1.0 toward a more reliable and versatile driving simulator. The upgraded base model offers finer-grained control over traffic semantics, including traffic lights and road markings, together with improved spatial consistency across cameras. Virtual City capabilities broaden scenario construction through a family of models conditioned on diverse 2D and 3D references, including support for generating multi-camera future videos from a single image. An enhanced post-training framework enables efficient causal, few-step generation with stronger temporal coherence, better quality retention over longer rollouts, and improved alignment with human preferences. We also introduce a dedicated evaluation suite to support systematic assessment and guide model iteration. Together, these upgrades expand the scenarios X-World can simulate and improve the efficiency and reliability of its generated observations, supporting broader use in autonomous-driving development.

Sat 31 OctArtificial Intelligence
The gist
Simulating virtual driving environments is hard because the scenarios must be realistic and controllable over time. The authors improved their X-World simulator to better control traffic elements like lights and road markings and to keep scenes consistent from multiple camera views. Their system can create new city layouts from simple inputs and generate future driving videos, helping test driving policies. They also made the simulator's output more stable and aligned with human preferences over longer simulations. A new evaluation suite helps check and improve the simulator’s performance systematically.
Open → 2611.00596v1

DomeGS improves virtual reality scenes by blending horizons and backgrounds

DomeGS: Perceptually Consistent Background Completion for Unbounded 3D Gaussian Splatting in Virtual Reality

Abstract: The growing need for high-quality 3D content remains among the most prevalent challenges in virtual reality (VR). Recently, novel view synthesis techniques have enabled rapid content creation, with 3D Gaussian Splatting (3DGS) emerging as one of the most promising methods for generating high-fidelity 3D scenes. However, using 3DGS to reconstruct unbounded outdoor scenes can leave horizons incomplete due to missing data or a lack of distant geometric constraints. Such incompleteness can break immersion, compromising VR's core characteristic. A typical workaround in traditional VR systems is a textured cube (skybox) or sphere (skydome) surrounding the environment to create the illusion of a distant background. However, these approaches struggle to blend seamlessly with the reconstructed 3DGS geometry. To address these issues, we introduce the DomeGS framework, which incorporates the skydome concept not as a flat background shell, but as a part of the reconstructed Gaussian Splatting 3D scene. By integrating the horizon and sky into the 3DGS representation, our method provides a unified, scene-consistent environment bridging foregrounds and backgrounds for greater realism. Experimental evaluation with N=36 users demonstrates that DomeGS significantly increases spatial presence (p=0.042) and realism (p=0.042) compared to the skybox baseline.

Tue 27 OctGraphics
The gist
Virtual reality often struggles to show outdoor scenes with distant horizons because 3D data can be incomplete far away. The authors developed DomeGS, which mixes the sky and horizon into the 3D scene itself rather than just adding flat background images. This makes VR environments feel more realistic and immersive by providing smoother transitions between the foreground and background. User tests showed DomeGS helped people feel more present and found the scenes more believable compared to traditional background methods.
Open → 2610.37172v1

Panorama generation method reduces distortion and improves seamless viewing

Squaring-the-Sphere: Distortion-Free and Seamless Panorama Generation via CubeAtlas

Abstract: Generating 360-degree panoramas requires both local perspective fidelity and global spherical continuity. Existing representations face a trade-off: equirectangular projection (ERP) enables seamless panorama modeling but suffers from severe distortion and a domain gap to pre-trained image generators, whereas cubemaps preserve perspective geometry but often produce inconsistent faces and visible seams. We present Squaring-the-Sphere, a panoramic image generation framework built on a novel representation CubeAtlas, which unfolds all cubemap faces into a single compact canvas. By arranging orientation-aligned, rotated polar regions alongside the equatorial faces, this representation preserves the perspective geometry of cubemaps while organizing the cross-face relationships required for seamless panorama assembly. It further enables a pretrained diffusion transformer to model the complete panorama jointly in a single pass, without altering the backbone architecture or positional encoding. While CubeAtlas has a token count comparable to ERP, its structured layout makes many dense attention interactions unnecessary. We therefore introduce a layout-aware sparse attention mask that preserves the interactions required for panorama continuity while removing geometrically irrelevant token pairs. As a result, CubeAtlas achieves better quality than ERP modeling with lower attention computation. Furthermore, we introduce a training-free super-resolution technique and a fusion scheme that enhance fine details in distortion-free perspective space before achieving the final ERP panorama. Experiments on various benchmarks demonstrate that our method improves geometric fidelity, cross-view consistency, and high-resolution visual quality over previous methods.

Thu 22 OctComputer Vision and Pattern Recognition
The gist
Making 360-degree panoramic images usually means choosing between two problems: either the image looks distorted or it has visible seams between parts. The authors created a new way to combine cube-shaped image pieces into one layout called CubeAtlas, which keeps the correct perspective and avoids seams. Their technique also uses smart computer attention to improve image quality while reducing the work needed. They also developed a method to sharpen the details without extra training. Their approach produces clear, consistent, and distortion-free panoramas.
Open → 2610.31565v1

Unravel creates movable 3D indoor scenes from a single photo

Unravel: Recovering an Interactive 3D Scene from a Single Image

Abstract: We present Unravel, a training-free framework that turns a single image into an interactive 3D scene, where every object is an independent and movable asset. Existing generators either produce one entangled geometry in which no object can be moved, or predict all objects in one feed-forward pass, where a wrong guess cannot be detected or repaired. Unravel instead takes a different approach: it first takes the scene apart, and then puts it back together. Starting from the input image, an agentic harness that iteratively peels the image layer by layer, and adopts point cloud as major modality for spatial relationship recovery. Specifically, a vision-language model plans which objects are fully visible, an image editor removes them and restores the occluded regions. Every edit is verified before being incorporated into the reconstruction. This process gradually turns occluded parts of the scene into direct observation, allowing objects to be reconstructed when they are best exposed rather than hallucinated from the original image alone. The scene is then rebuilt in reverse removal order, so that the supporting objects are naturally placed before the objects they support. Since a single-view image only reveals part of an object, we treat the observed points as a lower bound on its size: an asset may grow to cover its points, but can not shrink away from them. Finally, we physically settle the reconstructed scene to remove floating objects and interpenetrations, followed by an agentic audit that corrects remaining placement errors. Extensive experiments on diverse indoor scenes show that Unravel recovers more objects, places them more faithfully, and produces scenes that are more physically-plausible than existing methods.

Mon 12 OctComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Turning one picture of a room into a 3D scene where you can move every object is hard. Many methods either make a single mixed 3D shape or guess all objects at once, which can be wrong. The authors introduce Unravel, which peels off objects one by one, checks each step, and builds the scene back up with objects placed realistically. This method makes the 3D scene more accurate and lets you interact with individual objects.
Open → 2610.16146v1

Video world models learn internal memory retrieval for better scene consistency

Learning to Retrieve: Internalizing Memory Retrieval for Video World Models

Abstract: Video world models aim to generate explorable, 3D-consistent scene videos conditioned on camera trajectories. Existing approaches often rely on external memory systems that explicitly retrieve previously observed content to mitigate scene drift during long-horizon generation. However, these auxiliary memory pathways operate outside the model's internal generative dynamics, preventing the model from intrinsically learning when and what historical information should be retrieved. We propose to internalize memory retrieval into the generation process, allowing retrieval to emerge as an intrinsic behavior of the video world model rather than relying on an external memory system. Based on this principle, we introduce \textbf{Learning-to-Retrieve (L2R)}, which repurposes the model's persistent internal state as a memory for historical context. A camera-conditioned retrieval gate selectively accesses relevant historical information from this state, determining \textit{what to retrieve}, while a retrieval trigger determines \textit{when to retrieve}. We further supervise the trigger with a 3D re-visibility signal, activating retrieval when previously observed content re-enters the current view while otherwise preserving the existing context. Together, these components enable the model to intrinsically acquire memory retrieval behavior and incorporate relevant historical observations into generation without a separate retrieval pathway. Across multiple base models and camera-revisit benchmarks, L2R improves long-term scene consistency while eliminating the need for an external memory bank or 3D conditions. https://jkhu29.github.io/l2r

Thu 8 OctComputer Vision and Pattern Recognition
The gist
Generating consistent videos of 3D scenes over time is hard because models forget or drift away from earlier views. This work lets video models learn to remember past scenes by themselves, without separate memory components. The model figures out when to look back and what parts of the past to use based on the camera's position. This helps videos stay realistic and coherent for longer without needing external memory banks.
Open → 2610.11444v1

Interactive video storytelling improves by linking viewer intent and context

VIS-Ground: Video Interactive Storytelling with Contextual Grounding

Abstract: Video interactive storytelling enables viewers to actively steer how a video unfolds. However, once we allow viewers to intervene during generation, a new challenge arises: The viewer's request can have latent dependencies on both the grounding source and the current rendered video state. These dependencies may not be explicitly stated in any individual input, but emerge only when the source, rendered history, and new viewer intent are considered jointly. Existing interactive video generation systems primarily emphasize following viewer instructions, while source-grounded video generation methods focus on aligning generated content with an external narrative or knowledge source. This leaves a fundamental question underexplored: What context should a generation model ground on during interactive continuation, and how can heterogeneous, unstructured inputs be transformed into such grounding context? In this work, we formulate contextual grounding as the process of transforming heterogeneous input context into an executable constraint model for video generation. To address this challenge, we introduce VIS-Ground, which performs Structured Context Abstraction to recover grounded states and cross-context dependencies, Generation Constraints Induction to project relevant dependencies into candidate-specific constraints, and Constrained Video Generation to enforce these constraints through planning, verification, revision, and rendering. Across three video generation backbones, VIS-Ground consistently achieves the highest overall composite score, reaching an average absolute improvement of 10.3 points over the strongest per-backbone baselines. Detailed analysis further shows gains across both narrative and knowledge grounding, and reveals remaining challenges in dependency extraction, and faithful realization during video rendering.

Wed 7 OctComputer Vision and Pattern Recognition
The gist
Video interactive storytelling lets viewers influence how a story unfolds in a video. The challenge is that what a viewer wants next depends on both the story source and what’s already happened in the video, but these connections aren’t always obvious. The authors propose VIS-Ground, a system that turns all these mixed hints into clear instructions for video generation. This approach helps make videos that better follow the story and viewer requests, performing better than existing methods in tests.
Open → 2610.09326v1

Latent diffusion allows detailed local control of image style and content

Local Content-Style Control for Diffusion-based Image Stylization

Abstract: Image stylization with latent-diffusion models entangles two independently refined axes: what a region depicts and how it is depicted. Such pipelines expose only global controls, yet professional retouching demands deliberate, region-specific control. We lift two conditioning weights already present in a ControlNet + IP-Adapter stylization pipeline from global scalars to per-location spatial maps, yielding local, per-axis control of content and style in a single generative pass. Because the two weights act on disjoint pathways, adjusting them independently spans a 2x2 retouching vocabulary, from free regeneration to identity preservation. We validate that edits stay confined to the retouched region and that each weight predominantly steers its own axis. Our approach requires no retraining and drops unchanged into any such pipeline.

Tue 6 OctGraphicsComputer Vision and Pattern Recognition
The gist
Image stylization often changes the whole picture at once, making it hard to fix or improve just one part. The authors found a way to control how each part of an image looks and what it shows during style changes, all in one step. This system uses existing parts but lets users tweak settings for different areas, like changing the style in one area while keeping the content in another. This approach doesn’t need to retrain the model and works with current tools right away.
Open → 2610.08704v1

Egocentric gaze prediction improves with flow based trajectory modeling

GazeFlow: From Human Gaze Behavior to Generative Egocentric Gaze Prediction

Abstract: Egocentric gaze prediction enables many downstream applications but remains challenging, as human gaze is inherently stochastic. This stochasticity is constrained by structured temporal dynamics alternating between fixations and saccades, top-down influences from tasks, and bottom-up visual saliency. Based on this observation, we introduce GazeFlow, a framework that directly models gaze as a joint distribution of temporal gaze positions conditioned upon both top-down and bottom-up information. In particular, GazeFlow uses conditional flow matching (CFM): a learned velocity field iteratively transports a Gaussian noise sample into a plausible gaze trajectory drawn from this joint distribution. The velocity field is conditioned on bottom-up visual features extracted by a video encoder and on top-down task information obtained by globally querying these features. On standard datasets, GazeFlow achieves state-of-the-art performance on per-frame metrics, and the generated trajectories align better with human gaze temporal dynamics.

Tue 29 SeptComputer Vision and Pattern RecognitionMachine Learning
The gist
Predicting where people look when moving around is tricky because their eye movements change quickly and depend on both what they see and what they are trying to do. The authors created GazeFlow, a method that guesses not just single gaze points but whole gaze paths by combining visual details and task goals. It works by starting with random data and transforming it step-by-step into realistic eye movement patterns, matching how people actually look at things over time. Tested on common benchmarks, GazeFlow predicts eye positions more accurately than earlier methods.
Open → 2609.38519v1

Planning long journeys from random moves in unfamiliar environments

Learning to Plan from Random Exploration

Abstract: Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without policy-improvement training? Our random-walk analysis explains what temporal relations contain: short horizons reveal geodesic geometry in the diffusion limit, while longer horizons reveal connectivity between regions before mixing removes these distinctions. We learn these relations with a conditional energy-based model that estimates temporal log-density ratios through horizon-conditioned embeddings. The model is trained on observation pairs by noise-contrastive estimation, without action or reward labels. The planner queries these learned relations at different horizons as it moves toward the goal. At test time, a separate local dynamics model predicts candidate action outcomes, and the temporal model evaluates their progress toward the goal by selecting or aggregating estimated improvements across horizons. The agent executes one action and replans with both models fixed. Experiments demonstrate long-range maze planning from random exploration using states and images. Learned score fields, embedding probes, and planned routes exhibit properties of a multiscale cognitive map. We further demonstrate egocentric navigation from random exploration and manipulation planning from suboptimal data.

Tue 29 SeptMachine LearningRobotics
The gist
Planning a route in a new place usually needs practice toward a goal, but this research shows it can be done just by exploring randomly first. The authors explain how taking many random steps reveals the shape and connections of the space at different distances. They build a model that learns to understand these time-based relationships without needing information about actions or rewards. This model helps an agent plan actions step by step by predicting how each move would bring it closer to a target, even when starting from random wandering.
Open → 2609.38383v1

MUGEN dataset and Wan360 model enable interactive 360-degree video control

MUGEN: Interactive Panoramic World Exploration via Camera Control

Abstract: Interactive panoramic video generation aims to synthesize immersive 360\textdegree{} videos that remain visually coherent while following user-specified camera trajectories during exploration. However, progress is limited by a coupled data-and-model gap: existing panoramic video datasets are often short, weakly annotated, or lack camera trajectories, while existing camera-controlled video generation models are designed for perspective videos and do not directly support panoramic geometry. In this paper, we introduce MUGEN and Wan360 to address these limitations. MUGEN is a large-scale real-world panoramic video dataset tailored to interactive 360-degree world exploration, comprising over 1,300 hours of at least 4K panoramic videos with rich semantic and geometric annotations. Built on MUGEN, we further present Wan360, a camera-controllable interactive panoramic video generation model. Panoramic videos are commonly represented by EquiRectangular Projection (ERP), which unfolds a spherical 360-degree view into a rectangular frame with cyclic longitude seams and pole distortions. To this end, Wan360 introduces three parameter-free ERP-aware components: periodic longitude RoPE for seam-consistent positional encoding, ERP-aware padding for reducing boundary artifacts, and random roll yaw for consistent learning. For camera control, Wan360 uses a panoramic Plücker embedding that represents camera motion with ERP rays rather than perspective pinhole rays. Experiments show that MUGEN serves as a data foundation for panoramic world exploration, and that Wan360 enables high-quality, temporally coherent, camera-controllable 360-degree video generation.

Tue 29 SeptComputer Vision and Pattern Recognition
The gist
Interactive panoramic videos let users explore a full 360-degree world by moving a virtual camera, but making these videos smoothly follow user movements is hard. The authors created MUGEN, a large collection of real 360-degree videos with detailed labels to train computer models. They also built Wan360, a system that can generate new panoramic videos that respond to user-controlled camera paths while keeping visuals consistent and natural-looking. Wan360 solves special challenges from how 360-degree video maps a sphere to a flat image.
Open → 2609.38077v1

Adaptive action sequence transfer predicts user steps in new environments

Action Sequence Transfer via LLMs for Heterogeneous Environments

Abstract: We present an action sequence transfer system that adaptively transfers user action sequences across different target spaces. Given an input action sequence from a source space and scene graph representations of both the source and target environments, our system predicts a corresponding action sequence in the target space by adapting to the spatial and object constraints of the new environment. To achieve this, we leverage multi-level representations of user activity to generalize actions at varying levels of abstraction. To demonstrate our system, we collect a new scene graph-based dataset derived from the Ego4D GoalStep dataset for evaluation. Results indicate that our system can generate valid action sequences even between spaces with drastically different object configurations.

Mon 28 SeptRobotics
The gist
People often perform tasks step-by-step in one setting, but moving those exact steps to a different place with different objects can be tricky. The authors created a system that looks at the original actions and understands the objects and space involved, then predicts what actions would make sense in a new, different setting. They use detailed maps of the environments and ways to understand activities at different levels to help the system adapt. Their tests show the system can suggest reasonable action plans even when the new place looks very different.
Open → 2609.34730v1

Persona agent system adapts and evolves through long term interaction

Emergi-PersonaOS: A Persona Agent Operating System for Situational Adaptation and Controllable Evolution

Abstract: Symbiosis between humans and digital beings offers a vision for the future of human--machine interaction. In enduring human--machine relationships, personality provides a foundation for continuity of identity, individuality in interaction, and development through experience. We investigate this capacity through persona agents as computational implementations and introduce Emergi-PersonaOS, a psychology-grounded operating system for managing persona objects throughout their lifecycle. The system organizes dispositional traits, characteristic adaptations, and narrative identity into a three-layer persona representation, distinguishing relatively enduring persona beliefs from their activation in the current persona state. During situational adaptation, it integrates the current interlocutor, relationship, event, and retrieved memories to infer a persona state and generate actions and replies; during long-term development, it records experiences and outcomes, and develops and evaluates revision candidates through change attribution, meaning-making, and behavioral testing. Belief updates are managed through explicit review, traceable evidence and version records, and the ability to reject candidates, making persona evolution controllable. Using television-character dialogue as longitudinal material, we demonstrate long-horizon system operation and examine its principal mechanisms in a concrete implementation. This work provides a computational framework for persona agents to maintain individual continuity, produce situation-specific expression, and develop through experience over sustained interaction.

Wed 23 SeptArtificial Intelligence
The gist
People and digital helpers can build lasting relationships by having personalities that grow and change over time. The authors created Emergi-PersonaOS, a system that manages these personalities like a software operating system. It keeps track of traits, behaviors, and stories that shape the personality, and changes them based on interactions and memories. This allows digital agents to respond differently in situations while also learning and evolving in controlled ways over long interactions.
Open → 2609.27417v1

Visual parkour benchmark suite helps test robot locomotion in realistic scenes

The Neverwhere Visual Parkour Benchmark Suite

Abstract: State-of-the-art visual locomotion controllers are increasingly capable at handling complex visual environments, making evaluating their real-world performance before deployment increasingly difficult. This work intends to narrow this train/evaluation gap by developing a collection of hyper-photo-realistic, closed-loop evaluation environments - The Neverwhere Benchmark Suite - comprised of over sixty 3D Gaussian Splatting reconstructions of urban indoor and outdoor scenes. Our goal is to encourage large-scale and reproducible robot evaluation by making it easier to create and integrate Gaussian splats-based reconstructions into simulated continuous testing setups. We also underscore the potential pitfalls of relying exclusively on 3D Gaussian-generated data for training, by providing policy checkpoints trained over multiple Neverwhere scenes and their performance when evaluated in novel scenes. Our analysis illustrates the necessity of sourcing diverse data to ensure performance. Code and data are available on the project page: https://ziyc.github.io/neverwhere-bench/.

Mon 14 SeptRoboticsComputer Vision and Pattern RecognitionMachine Learning
The gist
It is hard to test how well robots move in complicated real-world places before sending them out. The authors created a set of super-realistic 3D scenes called the Neverwhere Benchmark Suite to help test robots’ movement controllers in virtual environments. These 3D scenes combine indoor and outdoor city spots made from a method called Gaussian Splatting. The authors also found that training robots using only these types of scenes can limit their ability to work well in new places, showing the need for varied data.
Open → 2609.16443v1