Text-Guided Vector Graphics Customization

Vector graphics are widely used in digital art and valued by designers for their scalability and layer-wise topological properties. However, the creation and editing of vector graphics necessitate creativity and design expertise, leading to a time-consuming process. In this paper, we propose a novel pipeline that generates high-quality customized vector graphics based on textual prompts while preserving the properties and layer-wise information of a given exemplar SVG. Our method harnesses the capabilities of large pre-trained text-to-image models. By fine-tuning the cross-attention layers of the model, we generate customized raster images guided by textual prompts. To initialize the SVG, we introduce a semantic-based path alignment method that preserves and transforms crucial paths from the exemplar SVG. Additionally, we optimize path parameters using both image-level and vector-level losses, ensuring smooth shape deformation while aligning with the customized raster image. We extensively evaluate our method using multiple metrics from vector-level, image-level, and text-level perspectives. The evaluation results demonstrate the effectiveness of our pipeline in generating diverse customizations of vector graphics with exceptional quality. The project page is https://intchous.github.io/SVGCustomization.

[1]  L. Sigal,et al.  Subpixel Deblurring of Anti‐Aliased Raster Clip‐Art , 2023, Comput. Graph. Forum.

[2]  Nupur Kumari,et al.  Multi-Concept Customization of Text-to-Image Diffusion , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Zhongliang Jing,et al.  CLIPVG: Text-Guided Image Manipulation Using Differentiable Vector Graphics , 2022, AAAI.

[4]  P. Abbeel,et al.  VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  M. Deisenroth,et al.  One-Shot Transfer of Affordance Regions? AffCorrs! , 2022, CoRL.

[6]  Yuanzhen Li,et al.  DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Amit H. Bermano,et al.  An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion , 2022, ICLR.

[8]  Y. Fu,et al.  Towards Layer-wise Image Vectorization , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Jean Oh,et al.  StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation , 2022, IJCAI.

[10]  Amit H. Bermano,et al.  CLIPasso , 2022, ACM Trans. Graph..

[11]  B. Ommer,et al.  High-Resolution Image Synthesis with Latent Diffusion Models , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Prafulla Dhariwal,et al.  GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models , 2021, ICML.

[13]  S. Bagon,et al.  Deep ViT Features as Dense Visual Descriptors , 2021, ArXiv.

[14]  Yingtao Tian,et al.  Modern Evolution Strategies for Creativity: Fitting Concrete Images and Abstract Concepts , 2021, EvoMUSART.

[15]  L. B. Soros,et al.  CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders , 2021, NeurIPS.

[16]  Julien Mairal,et al.  Emerging Properties in Self-Supervised Vision Transformers , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Ilya Sutskever,et al.  Learning Transferable Visual Models From Natural Language Supervision , 2021, ICML.

[18]  Alec Radford,et al.  Zero-Shot Text-to-Image Generation , 2021, ICML.

[19]  N. Mitra,et al.  Im2Vec: Synthesizing Vector Graphics without Vector Supervision , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Tzu-Mao Li,et al.  Differentiable vector graphics rasterization for editing and learning , 2020, ACM Trans. Graph..

[21]  Dinesh Manocha,et al.  P-cloth , 2020, ACM Trans. Graph..

[22]  Alla Sheffer,et al.  PolyFit : perception-aligned vectorization of raster clip-art via intermediate polygonal fitting , 2020 .

[23]  L. Sigal,et al.  PolyFit , 2020 .

[24]  Martin Jägersand,et al.  U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection , 2020, Pattern Recognit..

[25]  John Collomosse,et al.  Sketchformer: Transformer-Based Representation for Sketched Structure , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Philip H. S. Torr,et al.  Controllable Text-to-Image Generation , 2019, NeurIPS.

[27]  Alla Sheffer,et al.  Perception-driven semi-structured boundary vectorization , 2018, ACM Trans. Graph..

[28]  Adrien Bousseau,et al.  Photo2clipart , 2017, ACM Trans. Graph..

[29]  Douglas Eck,et al.  A Neural Representation of Sketch Drawings , 2017, ICLR.

[30]  Bernt Schiele,et al.  Generative Adversarial Text to Image Synthesis , 2016, ICML.

[31]  Dani Lischinski,et al.  Depixelizing pixel art , 2011, ACM Trans. Graph..

[32]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Paul Harris,et al.  The Fundamentals of Graphic Design , 2008 .

[34]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[35]  Antoine Quint,et al.  Scalable Vector Graphics , 2020, Definitions.

[36]  James Richard. Diebel,et al.  Bayesian image vectorization : the probabilistic inversion of vector image rasterization / james richard diebel. , 2008 .

[37]  P. Selinger Potrace : a polygon-based tracing algorithm , 2003 .

[38]  Jian Sun,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Effective Clipart Image Vectorization through Direct Optimization of Bezigons , 2022 .