Neural Strokes: Stylized Line Drawing of 3D Shapes

This paper introduces a model for producing stylized line drawings from 3D shapes. The model takes a 3D shape and a viewpoint as input, and outputs a drawing with textured strokes, with variations in stroke thickness, deformation, and color learned from an artist’s style. The model is fully differentiable. We train its parameters from a single training drawing of another 3D shape. We show that, in contrast to previous image-based methods, the use of a geometric representation of 3D shape and 2D strokes allows the model to transfer important aspects of shape and texture style while preserving contours. Our method outputs the resulting drawing in a vector representation, enabling richer downstream analysis or editing in interactive applications. Our code and dataset are available at our project page: www.github.com/DifanLiu/NeuralStrokes

[1]  Derek Nowrouzezahrai,et al.  Data-driven curvature for real-time line drawing of dynamic scenes , 2009, ACM Trans. Graph..

[2]  Frédo Durand,et al.  Programmable rendering of line drawing from 3D scenes , 2010, TOGS.

[3]  Adam Finkelstein,et al.  Suggestive contours for conveying shape , 2003, ACM Trans. Graph..

[4]  Hans-Peter Seidel,et al.  Ridge-Valley Lines on Meshes via Implicit Surface Fitting , 2004 .

[5]  Jaroslav Nesetril.  Art of Drawing , 1999, Graph Drawing.

[6]  Adam Finkelstein,et al.  Line drawings from 3D models , 2005, SIGGRAPH '08.

[7]  Alexei A. Efros,et al.  The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  A. L. Guptill.  Rendering in pen and ink , 1976 .

[10]  Eli Shechtman,et al.  Im2Pencil: Controllable Pencil Illustration From Photographs , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Li Fei-Fei,et al.  Perceptual Losses for Real-Time Style Transfer and Super-Resolution , 2016, ECCV.

[12]  Alla Sheffer,et al.  Lifting freehand concept sketches into 3D , 2020, ACM Trans. Graph..

[13]  Tzu-Mao Li,et al.  Differentiable vector graphics rasterization for editing and learning , 2020, ACM Trans. Graph..

[14]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[15]  Alexei A. Efros,et al.  Contrastive Learning for Unpaired Image-to-Image Translation , 2020, ECCV.

[16]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Hans-Peter Seidel,et al.  Ridge-valley lines on meshes via implicit surface fitting , 2004, ACM Trans. Graph..

[18]  Serge J. Belongie,et al.  Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[19]  David Salesin,et al.  Rendering parametric surfaces in pen and ink , 1996, SIGGRAPH.

[20]  Patrick Coleman,et al.  Making Souls: Methods and a Pipeline for Volumetric Characters , 2020, SIGGRAPH Talks.

[21]  Stephen DiVerdi,et al.  HelpingHand: example-based stroke stylization , 2012, ACM Trans. Graph..

[22]  Sepp Hochreiter,et al.  GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium , 2017, NIPS.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Eli Shechtman,et al.  StyLit , 2016, ACM Trans. Graph..

[25]  Daniel Cohen-Or,et al.  SketchPatch , 2020, ACM Trans. Graph..

[26]  H. Reeves,et al.  The Guild handbook of scientific illustration , 1991 .

[27]  T. Funkhouser,et al.  How well do line drawings depict shape? , 2009, SIGGRAPH '09.

[28]  Stephen DiVerdi,et al.  RealBrush: painting with examples of physical media , 2013, ACM Trans. Graph..

[29]  Nuria Oliver,et al.  Curve Analogies , 2002, Rendering Techniques.

[30]  Marc Alexa,et al.  ABC: A Big CAD Model Dataset for Geometric Deep Learning , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Andrea Vedaldi,et al.  Instance Normalization: The Missing Ingredient for Fast Stylization , 2016, ArXiv.

[32]  Aaron Hertzmann.  Why Do Line Drawings Work? A Realism Hypothesis , 2020, Perception.

[33]  Ming-Hsuan Yang,et al.  Universal Style Transfer via Feature Transforms , 2017, NIPS.

[34]  Oriol Vinyals,et al.  Synthesizing Programs for Images using Reinforced Adversarial Learning , 2018, ICML.

[35]  Aaron Hertzmann,et al.  Isophote distance: a shading approach to artistic stroke thickness , 2007, NPAR '07.

[36]  Pierre Bénard,et al.  Stylizing animation by example , 2013, ACM Trans. Graph..

[37]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[38]  David Salesin,et al.  Image Analogies , 2001, SIGGRAPH.

[39]  David Salesin,et al.  Computer-generated pen-and-ink illustration , 1994, SIGGRAPH.

[40]  Frédo Durand,et al.  Apparent ridges for line drawing , 2007, ACM Trans. Graph..

[41]  Adam Finkelstein,et al.  WYSIWYG NPR: drawing strokes directly on 3D models , 2002, SIGGRAPH.

[42]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Douglas DeCarlo,et al.  Depicting 3D shape using lines , 2012, Electronic Imaging.

[44]  Aaron Hertzmann,et al.  Neural Contours: Learning to Draw Lines From 3D Shapes , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.