Digital marbling programs recovered efficiently from images with replay method
Inverse Digital Marbling: Recovering Gesture Programs with a Replay Adjoint
GraphicsComputer Vision and Pattern Recognition
Summary
Paper marbling creates patterns by dropping pigments on water, where new drops change earlier ones, making it tricky to recreate or edit digitally. The authors present a method that finds a step-by-step digital 'program' of gestures to recreate a marbled image using a special mathematical replay technique. This technique saves memory and time compared to traditional methods, allowing the digital marbling process to be edited and replayed accurately. The recovered programs work well across different image resolutions and allow certain types of edits, but not all image-based changes.
What this means in practice
- •For digital artists: Create editable digital marbling patterns from images for customized artwork generation and modification.
- •For computer graphics developers: Integrate efficient inverse digital marbling program extraction to enable programmatic editing of texture and fluid-like effects in rendering tools.
Authors
Tianqi Liu, Yushan Han, Hang Liu
Abstract
Pigment deposition in paper marbling displaces the pattern already present, coupling the appearance of each gesture to later actions. We recover executable programs for a deposition-based digital marbling model: given a target image, we optimise an ordered program of capsule insertions whose replay approximates it. The capsule primitive continuously joins circular drops to elongated deposits. Its transport is exactly area-preserving and has a closed-form inverse on the exterior of the deposited region. A replay adjoint reconstructs intermediate states, retaining coordinates lost inside deposits and periodic position checkpoints. At 2000 gestures and 1024^2 pixels, the PyTorch replay implementation uses 8.7x less memory than the tested checkpointed-autograd configuration at comparable step time; the fused implementation fits a program in about four minutes on one workstation GPU. We evaluate image reconstruction on five marbled sheets, compare against transport-disabled fitting, one-pass geometric compensation and a published stroke-based fitter at matched stroke count, and measure sensitivity to an alternative ordered-drop transport. Recovered programs replay across a 4x range of linear resolution. Edits specified in program order or in palette space -- inserting a gesture, recolouring a stage, translating a stage -- replay correctly under the same model; edits specified by image content, such as moving a motif, do not. On synthetic targets with known generating programs, the recovered programs match the images but not the generating gestures under a positional matching statistic. The output is an editable program in the specified digital medium.