StrokeAggregator: consolidating raw sketches into artist-intended curve drawings

When creating line drawings, artists frequently depict intended curves using multiple, tightly clustered, or overdrawn, strokes. Given such sketches, human observers can readily envision these intended, aggregate, curves, and mentally assemble the artist's envisioned 2D imagery. Algorithmic stroke consolidation---replacement of overdrawn stroke clusters by corresponding aggregate curves---can benefit a range of sketch processing and sketch-based modeling applications which are designed to operate on consolidated, intended curves. We propose StrokeAggregator, a novel stroke consolidation method that significantly improves on the state of the art, and produces aggregate curve drawings validated to be consistent with viewer expectations. Our framework clusters strokes into groups that jointly define intended aggregate curves by leveraging principles derived from human perception research and observation of artistic practices. We employ these principles within a coarse-to-fine clustering method that starts with an initial clustering based on pairwise stroke compatibility analysis, and then refines it by analyzing interactions both within and in-between clusters of strokes. We facilitate this analysis by computing a common 1D parameterization for groups of strokes via common aggregate curve fitting. We demonstrate our method on a large range of line drawings, and validate its ability to generate consolidated drawings that are consistent with viewer perception via qualitative user evaluation, and comparisons to manually consolidated drawings and algorithmic alternatives.

[1]  K. Sasaki,et al.  Learning to simplify , 2016, ACM Trans. Graph..

[2]  Alla Sheffer,et al.  Modeling Character Canvases from Cartoon Drawings , 2015, ACM Trans. Graph..

[3]  Adrien Bousseau,et al.  SketchSoup: Exploratory Ideation Using Design Sketches , 2017, Comput. Graph. Forum.

[4]  Ravin Balakrishnan,et al.  ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models , 2008, UIST '08.

[5]  Alla Sheffer,et al.  Gesture3D: posing 3D characters via gesture drawings , 2016, ACM Trans. Graph..

[6]  Adam Finkelstein,et al.  Coherent stylized silhouettes , 2003, ACM Trans. Graph..

[7]  Adrien Bousseau,et al.  CrossShade: shading concept sketches using cross-section curves , 2012, ACM Trans. Graph..

[8]  Pascal Barla,et al.  Non‐Oriented MLS Gradient Fields , 2013, Comput. Graph. Forum.

[9]  Cindy Grimm,et al.  Just DrawIt: a 3D sketching system , 2012, SBIM '12.

[10]  Daniel Cohen-Or,et al.  Conjoining Gestalt rules for abstraction of architectural drawings , 2011, ACM Trans. Graph..

[11]  In-Kwon Lee,et al.  Curve reconstruction from unorganized points , 2000, Comput. Aided Geom. Des..

[12]  Pengfei Xu,et al.  Lazy selection , 2012, ACM Trans. Graph..

[13]  Adrien Bousseau,et al.  Fidelity vs. simplicity , 2016, ACM Trans. Graph..

[14]  Levent Burak Kara,et al.  Beautification of Design Sketches Using Trainable Stroke Clustering and Curve Fitting , 2011, IEEE Transactions on Visualization and Computer Graphics.

[15]  D. Field,et al.  Integration of contours: new insights , 1999, Trends in Cognitive Sciences.

[16]  Pascal Barla,et al.  Geometric clustering for line drawing simplification , 2005, SIGGRAPH '05.

[17]  Philip J. Farrugia,et al.  Scribbles to Vectors: Preparation of Scribble Drawings for CAD Interpretation , 2007, SBIM.

[18]  Markus H. Gross,et al.  Smart Scribbles for Sketch Segmentation , 2012, Comput. Graph. Forum.

[19]  Frédo Durand,et al.  Density measure for line-drawing simplification , 2004, 12th Pacific Conference on Computer Graphics and Applications, 2004. PG 2004. Proceedings..

[20]  Karl Tombre,et al.  Robust and accurate vectorization of line drawings , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  John Snyder,et al.  Freeform vector graphics with controlled thin-plate splines , 2011, ACM Trans. Graph..

[22]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[23]  Arthur Zimek,et al.  Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection , 2015, ACM Trans. Knowl. Discov. Data.

[24]  Baoquan Chen,et al.  Efficient and Dynamic Simplification of Line Drawings , 2008, Comput. Graph. Forum.

[25]  Karan Singh,et al.  Sketching piecewise clothoid curves , 2008, SBM'08.

[26]  Chi-Wing Fu,et al.  Globally Consistent Wrinkle-Aware Shading of Line Drawings , 2018, IEEE Transactions on Visualization and Computer Graphics.

[27]  S. Palmer,et al.  A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. , 2012, Psychological bulletin.

[28]  Anthony Wirth,et al.  Correlation Clustering , 2010, Encyclopedia of Machine Learning and Data Mining.

[29]  Jaakko Lehtinen,et al.  Sketching Clothoid Splines Using Shortest Paths , 2010, Comput. Graph. Forum.

[30]  D. Levin,et al.  Mesh-Independent Surface Interpolation , 2004 .

[31]  Adrien Bousseau,et al.  True2Form: 3D curve networks from 2D sketches via selective regularization , 2014, ACM Trans. Graph..

[32]  Gongning Luo,et al.  A graph-based method for fitting planar B-spline curves with intersections , 2016, J. Comput. Des. Eng..

[33]  Thomas Baudel,et al.  A mark-based interaction paradigm for free-hand drawing , 1994, UIST '94.

[34]  Hongbo Fu,et al.  Vectorizing line drawings with near-constant line width , 2012, 2012 19th IEEE International Conference on Image Processing.

[35]  Hiroshi Ishikawa,et al.  Mastering Sketching , 2017, ACM Trans. Graph..

[36]  Koos Eissen,et al.  Sketching: Drawing Techniques for Product Designers , 2009 .

[37]  Thomas Brox,et al.  Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts , 2015, ICCV.

[38]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, SIGGRAPH '05.

[39]  Hod Lipson,et al.  Optimization-based reconstruction of a 3D object from a single freehand line drawing , 1996, Comput. Aided Des..

[40]  Markus H. Gross,et al.  Topology-driven vectorization of clean line drawings , 2013, ACM Trans. Graph..

[41]  Paul L. Rosin Grouping Curved Lines , 1994, BMVC.

[42]  Kwan-Liu Ma,et al.  Rendering complexity in computer-generated pen-and-ink illustrations , 2004, NPAR '04.

[43]  Tien-Tsin Wong,et al.  Closure-aware sketch simplification , 2015, ACM Trans. Graph..