A survey of sketch based modeling systems

As 3D technology, including computer graphics, virtual reality and 3D printing, has been rapidly developed in the past years, 3D models are gaining an increasingly huge demand. Traditional 3D modeling platforms such as Maya and ZBrush, utilize “windows, icons, menus, pointers” (WIMP) interface paradigms for fine-grained control to construct detailed models. However, the modeling progress can be tedious and frustrating and thus too hard for a novice user or even a well trained artist. Therefore, a more intuitive interface is needed. Sketch, an intuitive communication and modeling tool for human beings, becomes the first choice of modeling community. So far, various sketch-based modeling systems have been created and studied. In this paper, we attempt to show how these systems work and give a comprehensive survey.We review and categorize the systems in four aspects: the input, the knowledge they use, the modeling approach and the output. We also discuss about inherent challenges and open problems for researchers in the future.

[1]  Daniel Cohen-Or,et al.  Sketch‐to‐Design: Context‐Based Part Assembly , 2012, Comput. Graph. Forum.

[2]  Marc Alexa,et al.  A sketch-based interface for detail-preserving mesh editing , 2005, SIGGRAPH 2005.

[3]  John F. Hughes,et al.  SmoothSketch: 3D free-form shapes from complex sketches , 2006, SIGGRAPH '06.

[4]  Vladimir G. Kim,et al.  Data‐Driven Shape Analysis and Processing , 2015, Comput. Graph. Forum.

[5]  Christian Rössl,et al.  Laplacian surface editing , 2004, SGP '04.

[6]  Frédo Durand,et al.  3D modeling with silhouettes , 2010, SIGGRAPH 2010.

[7]  Tao Ju,et al.  Topology-constrained surface reconstruction from cross-sections , 2015, ACM Trans. Graph..

[8]  Ravin Balakrishnan,et al.  EverybodyLovesSketch: 3D sketching for a broader audience , 2009, UIST '09.

[9]  Leonidas J. Guibas,et al.  Estimating image depth using shape collections , 2014, ACM Trans. Graph..

[10]  Songde Ma,et al.  A sketch-based interactive framework for real-time mesh segmentation , 2007 .

[11]  Wenping Wang,et al.  Flow aligned surfacing of curve networks , 2015, ACM Trans. Graph..

[12]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[13]  Arvin Agah,et al.  A survey of sketch-based 3-D modeling techniques , 2009, Interact. Comput..

[14]  Daniel Cohen-Or,et al.  Flower reconstruction from a single photo , 2014, Comput. Graph. Forum.

[15]  Donald D. Hoffman.  Visual Intelligence: How We Create What We See , 1998 .

[16]  John F. Hughes,et al.  A Sketch-Based Interface for Clothing Virtual Characters , 2007, IEEE Computer Graphics and Applications.

[17]  Pierre Alliez,et al.  Geometric Modeling Based on Polygonal Meshes , 2008, Eurographics.

[18]  Ligang Liu,et al.  Modeling by Drawing with Shadow Guidance , 2013, Comput. Graph. Forum.

[19]  Dieter Fox,et al.  RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments , 2012, Int. J. Robotics Res..

[20]  Adrien Bousseau,et al.  BendFields , 2015, ACM Trans. Graph..

[21]  Donald D. Hoffman,et al.  Visual Intelligence: How We Create What We See , 1998 .

[22]  Takeo Igarashi,et al.  Structured annotations for 2D-to-3D modeling , 2009, ACM Trans. Graph..

[23]  Michael Garland,et al.  Sketching mesh deformations , 2005, SI3D.

[24]  J. Hughes,et al.  SmoothSketch: 3D free-form shapes from complex sketches , 2006, ACM Trans. Graph..

[25]  Loong Fah Cheong,et al.  Symmetric architecture modeling with a single image , 2009, ACM Trans. Graph..

[26]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[27]  Tinne Tuytelaars,et al.  Sketch classification and classification-driven analysis using Fisher vectors , 2014, ACM Trans. Graph..

[28]  Szymon Rusinkiewicz,et al.  Modeling by example , 2004, SIGGRAPH 2004.

[29]  Steven Skiena,et al.  Visualizing Objects with Mirrors , 2004, Comput. Graph. Forum.

[30]  Ronald N. Perry,et al.  Kizamu: a system for sculpting digital characters , 2001, SIGGRAPH.

[31]  Mario Costa Sousa,et al.  Sketch-based modeling: A survey , 2009, Comput. Graph..

[32]  Takeo Igarashi,et al.  Magic canvas: interactive design of a 3-D scene prototype from freehand sketches , 2007, GI '07.

[33]  Xuejin Chen,et al.  Sketch-based tree modeling using Markov random field , 2008, ACM Trans. Graph..

[34]  Ligang Liu,et al.  Photo-inspired model-driven 3D object modeling , 2011, ACM Trans. Graph..

[35]  Yong Jae Lee,et al.  ShadowDraw: real-time user guidance for freehand drawing , 2011, ACM Trans. Graph..

[36]  Marc Alexa,et al.  Sketch-Based Image Retrieval: Benchmark and Bag-of-Features Descriptors , 2011, IEEE Transactions on Visualization and Computer Graphics.

[37]  Marc Alexa,et al.  Sketching contours , 2008, Comput. Graph..

[38]  Ping Tan,et al.  Symmetric architecture modeling with a single image , 2009, SIGGRAPH 2009.

[39]  Alla Sheffer,et al.  Modeling from contour drawings , 2009, SBIM '09.

[40]  Xuejin Chen,et al.  Sketch-based tree modeling using Markov random field , 2008, SIGGRAPH 2008.

[41]  Sanghun Nam,et al.  SPACESKETCH: Shape modeling with 3D meshes and control curves in stereoscopic environments , 2012, Comput. Graph..

[42]  Craig Gotsman,et al.  ArtiSketch: A System for Articulated Sketch Modeling , 2013, Comput. Graph. Forum.

[43]  Marc Alexa,et al.  Differential coordinates for local mesh morphing and deformation , 2003, The Visual Computer.

[44]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[45]  Harry Shum,et al.  Sketching reality: Realistic interpretation of architectural designs , 2008, TOGS.

[46]  Ravin Balakrishnan,et al.  ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models , 2008, UIST '08.

[47]  Marc Alexa,et al.  SilSketch: automated sketch-based editing of surface meshes , 2007, SBIM '07.

[48]  Marc Alexa,et al.  FiberMesh: designing freeform surfaces with 3D curves , 2007, SIGGRAPH 2007.

[49]  David Salesin,et al.  A sketching interface for articulated figure animation , 2006, SIGGRAPH 2006.

[50]  Brian Wyvill,et al.  ShapeShop: sketch-based solid modeling with BlobTrees , 2007, SIGGRAPH '07.

[51]  Adrien Bousseau,et al.  True2Form: 3D curve networks from 2D sketches via selective regularization , 2014, ACM Trans. Graph..

[52]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[53]  Suguru Saito,et al.  Single-view sketch based modeling , 2011, SBIM '11.

[54]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[55]  Sangwon Lee,et al.  A Sketch‐Based User Interface for Reconstructing Architectural Drawings , 2008, Comput. Graph. Forum.

[56]  Daniel Cohen-Or,et al.  3-Sweep , 2013, ACM Trans. Graph..

[57]  Daniel Cohen-Or,et al.  Geosemantic Snapping for Sketch‐Based Modeling , 2013, Comput. Graph. Forum.

[58]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision: Research Topics and Applications , 2012 .

[59]  Ismail Khalid Kazmi,et al.  Efficient sketch‐based creation of detailed character models through data‐driven mesh deformations , 2015, Comput. Animat. Virtual Worlds.

[60]  Long Quan.  Image-Based Modeling , 2009, accv 2009.

[61]  Christian Rössl,et al.  Geometric modeling based on polygonal meshes Video files associated with this course are available from the citation page , 2007, SIGGRAPH Courses.

[62]  Yong Jae Lee,et al.  ShadowDraw: real-time user guidance for freehand drawing , 2011, SIGGRAPH 2011.

[63]  Karan Singh,et al.  SecondSkin: sketch-based construction of layered 3D models , 2015, ACM Trans. Graph..

[64]  Ryan Schmidt,et al.  Analytic drawing of 3D scaffolds , 2009, SIGGRAPH 2009.

[65]  Hujun Bao,et al.  Interactive localized liquid motion editing , 2013, ACM Trans. Graph..

[66]  Jianxiong Xiao,et al.  Single image tree modeling , 2008, SIGGRAPH 2008.

[67]  Joseph A. Paradiso,et al.  The Hybrid Artisans: A Case Study in Smart Tools , 2013, TCHI.

[68]  Xiaoru Yuan,et al.  Sketch-based Segmentation of Scanned Outdoor Environment Models , 2005, SBM.

[69]  Shi-Min Hu,et al.  Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models , 2013, ACM Trans. Graph..

[70]  Lance R. Williams,et al.  Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience , 1997, Neural Computation.