Foresee shows what actions enable with clear visual previews
Foresee: Making Procedural Futures Tangible through Generative Previews
Human-Computer Interaction
Summary
People following step-by-step instructions often miss understanding what their actions lead to later. The authors created Foresee, a tool that not only tells users the next step but also shows previews of what might happen afterward. It uses a mix of rendered images, edits, and videos to make these previews clear and helpful. Studies with people showed Foresee helped users complete tasks more accurately and remember later steps better compared to other guidance methods. The authors also found that showing previews helps users understand and trust instructions more than just symbolic or ambiguous hints.
What this means in practice
- •For digital artists: Improve tutorial systems by showing what future steps achieve during digital painting workflows.
- •For game designers: Integrate step guidance with visual previews to help players better understand and execute game actions.
Authors
Yusi Sun, Ying Jiang, Jiayin Lu, Jing Qian, Yin Yang, Yong-Hong Kuo, Chenfanfu Jiang
Abstract
Procedural guidance can show users how to act without explaining what an action enables later. We present Foresee, an interface combining next-step guidance with consequence previews, method comparisons, and cross-step access. A hybrid pipeline combines structured rendering, image editing, and video generation. A literature-informed framework and six-participant formative study shaped its design. A 24-participant study compared AR guidance, AR with generated video, and Foresee across digital painting, gameplay, and cocktail making. Exploratory painting comparisons showed higher correct completion without a recorded error than AR and higher post-task quiz accuracy than generated-video guidance under task-wise correction. Interviews revealed that users could overlook valued explanations, match visible outcomes without understanding later possibilities, or wait for concrete demonstrations when symbolic cues were unclear. These findings inform generative previews that explain what actions enable or constrain while supporting ongoing execution.