Scene text editing that keeps original font colors and style
STEPS: Scene Text Editing with Preserved Style Using Diffusion and Contrastive Style Encoding
Computer Vision and Pattern RecognitionMachine Learning
Summary
Changing text in pictures while keeping the same look is hard. The authors created STEPS, a new method that changes words in images but keeps the style, like font and colors, unchanged. They made a special tool to understand the style separately from the letters, helping make the changed text look natural. Their method works better than older ones in keeping style, being easy to read, and looking good.
What this means in practice
- •For graphic designers: Replace text in product images while maintaining exact font and color styles for branding consistency.
- •For digital marketing teams: Create tailored advertisements by editing text in visuals without losing original design style.$Commercial implications: Enables customized ad visuals with consistent brand identity for marketing services selling personalized content creation.
Authors
Nicolas Thiebaut, Nameer Hirschkind, Xiao Yu, Kyle Spence
Abstract
We introduce Scene Text Editing with Preserved Style (STEPS), a novel diffusion model architecture for quality text replacement in images. Scene Text Editing (STE), also known as Visual Text Editing, consists of changing the textual content in an image while conserving the original style, e.g. font, colors, orientation, background, etc. STEPS advances the state of the art in STE through directed focus on improved style preservation. We introduce a style encoder for visual text that captures style independently of textual content, and a model architecture that combines the style encoder with multiple semantic conditions (target text characters encoding and rendered glyphs). STEPS achieves superior results to previous STE methods in style preservation, output readability, and subjective quality.