A Taxonomy of Modeling Techniques using Sketch-Based Interfaces

Traditional user interfaces in modeling have followed the WIMP (Window, Icon, Menu, Pointer) paradigm. While functional and powerful, they can also be cumbersome and daunting to a novice user; creating a complex model requires much expertise and effort. A recent trend is toward more accessible and natural interfaces, which has lead to sketch-based interfaces for modeling (SBIM). The goal is to allow hand-drawn sketches to be used in the modeling process, from rough model creation through to fine detail construction. Mapping 2D sketches to a 3D modeling operation is a difficult and ambiguous task, so our categorization is based on how an SBIM application chooses to interpret a sketch, of which there are three primary methods: to create a 3D model, to add details to an existing model, or to deform and manipulate a model. In this STAR, we present a taxonomy of sketchbased interfaces focused on geometric modeling applications. The canonical and recent works are presented and classified, including techniques for sketch acquisition, filtering, and interpretation. The report also includes a discussion of important challenges and open problems for researchers to tackle in the coming years.

[1]  Hod Lipson,et al.  A sketch-based interface for iterative design and analysis of 3D objects , 2007, SBM.

[2]  Donald D. Hoffman,et al.  Salience of visual parts , 1997, Cognition.

[3]  Ada Wai-Chee Fu,et al.  Efficient time series matching by wavelets , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[4]  Chen Yang,et al.  Sketch-based modeling of parameterized objects , 2005, SIGGRAPH '05.

[5]  Marc Alexa,et al.  FiberMesh: designing freeform surfaces with 3D curves , 2007, ACM Trans. Graph..

[6]  Marc Alexa,et al.  SilSketch: automated sketch-based editing of surface meshes , 2007, SBIM '07.

[7]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[8]  Michael Garland,et al.  Sketching mesh deformations , 2005, ACM Trans. Graph..

[9]  Henning Biermann,et al.  Sharp Features on Multiresolution Subdivision Surfaces , 2002, Graph. Model..

[10]  Takeo Igarashi,et al.  Smooth meshes for sketch-based freeform modeling , 2003, I3D '03.

[11]  Joaquim A. Jorge,et al.  BlobMaker: Free form Modelling with Variational Implicit Surfaces , 2003 .

[12]  Levent Burak Kara,et al.  Construction and modification of 3D geometry using a sketch-based interface , 2006, SBM'06.

[13]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[14]  Manuel Contero,et al.  CIGRO: A Minimal Instruction Set Calligraphic Interface for Sketch-Based Modeling , 2003, ICCSA.

[15]  James A. Landay,et al.  Sketching Interfaces: Toward More Human Interface Design , 2001, Computer.

[16]  Michiel van de Panne,et al.  Constellation models for sketch recognition , 2006, SBM'06.

[17]  Vincent Hayward,et al.  Haptic interfaces and devices , 2004 .

[18]  P. Giblin.  Computational geometry: algorithms and applications (2nd edn.), by M. de Berg, M. van Kreveld, M. Overmars and O. Schwarzkopf. Pp. 367. £20.50. 2000. ISBN 3 540 65620 0 (Springer-Verlag). , 2001, The Mathematical Gazette.

[19]  Michiel van de Panne,et al.  Motion doodles: an interface for sketching character motion , 2004, SIGGRAPH Courses.

[20]  Joseph J. LaViola,et al.  Sketching and gestures 101 , 2007, SIGGRAPH '07.

[21]  James F. O'Brien,et al.  Variational Implicit Surfaces , 1999 .

[22]  Christos Faloutsos,et al.  Efficient Similarity Search In Sequence Databases , 1993, FODO.

[23]  Ivan E. Sutherland,et al.  Sketchpad a Man-Machine Graphical Communication System , 1899, Outstanding Dissertations in the Computer Sciences.

[24]  Xiaoru Yuan,et al.  Sketch-based Segmentation of Scanned Outdoor Environment Models , 2005, SBM.

[25]  Pedro Santos,et al.  Stroke-input methods for immersive styling environments , 2004, Proceedings Shape Modeling Applications, 2004..

[26]  Brian Wyvill,et al.  Improving the Sketch-Based Interface Forming Curves from Many Small Strokes , 2007 .

[27]  Brian Wyvill,et al.  ShapeShop: sketch-based solid modeling with BlobTrees , 2006, SBM.

[28]  D. F. Rogers.  Constrained B-spline curve and surface fitting , 1989 .

[29]  Mario Costa Sousa,et al.  Sketch-based modeling with few strokes , 2005, SCCG '05.

[30]  Joseph Ross Mitchell,et al.  Sketch-based volumetric seeded region growing , 2006, SBM'06.

[31]  Mario Costa Sousa,et al.  Transformation strokes , 2006, SBM'06.

[32]  Abhishek Ranjan,et al.  A suggestive interface for image guided 3D sketching , 2004, CHI.

[33]  Lee Markosian,et al.  Free-form Sketch , 2007, SBIM '07.

[34]  Katsuhiko Onishi,et al.  Modeling of Trees with Interactive L-System and 3D Gestures , 2006, BioADIT.

[35]  Baoquan Chen,et al.  SMARTPAPER: An Interactive and User Friendly Sketching System , 2004, Comput. Graph. Forum.

[36]  Takeo Igarashi,et al.  Magic canvas: interactive design of a 3-D scene prototype from freehand sketches , 2007, GI '07.

[37]  Yoshisuke Kurozumi,et al.  Polygonal approximation by the minimax method , 1982, Comput. Graph. Image Process..

[38]  Remco C. Veltkamp,et al.  Shape matching: similarity measures and algorithms , 2001, Proceedings International Conference on Shape Modeling and Applications.

[39]  James A. Landay,et al.  Visual similarity of pen gestures , 2000, CHI.

[40]  Nezam Mahdavi-Amiri,et al.  A filtered b-spline models of scanned digital images , 1998 .

[41]  Manuel Contero,et al.  On the evolution of geometrical reconstruction as a core technology to sketch-based modeling , 2004, SBM'04.

[42]  Levent Burak Kara,et al.  Pen-based styling design of 3D geometry using concept sketches and template models , 2006, SPM '06.

[43]  Satoshi Matsuoka,et al.  Interactive beautification: a technique for rapid geometric design , 2006, SIGGRAPH Courses.

[44]  Donald D. Hoffman,et al.  Visual Intelligence: How We Create What We See , 1998 .

[45]  Mario Costa Sousa,et al.  Modeling plant structures using concept sketches , 2006, NPAR.

[46]  Adam Finkelstein,et al.  Suggestive contours for conveying shape , 2003, ACM Trans. Graph..

[47]  Shehzad Khalid,et al.  Motion Trajectory Learning in the DFT-Coefficient Feature Space , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[48]  Allen R. Hanson,et al.  Perceptual completion of occluded surfaces , 1996, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Joaquim A. Jorge,et al.  Towards Calligraphic Interfaces: Sketching 3D Scenes with Gestures and Context Icons , 2000, WSCG.

[50]  Mario Costa Sousa,et al.  Fast stroke matching by angle quantization , 2007, IMMERSCOM.

[51]  Randall Davis,et al.  LADDER, a sketching language for user interface developers , 2005, Comput. Graph..

[52]  Hongxin Zhang,et al.  Prototype Modeling from Sketched Silhouettes based on Convolution Surfaces , 2004, Comput. Graph. Forum.

[53]  Alla Sheffer,et al.  Virtual Garments: A Fully Geometric Approach for Clothing Design , 2006, Comput. Graph. Forum.

[54]  Tomoyuki Nishita,et al.  Character animation creation using hand-drawn sketches , 2005, The Visual Computer.

[55]  Bo Yu.  Recognition of freehand sketches using mean shift , 2003, IUI '03.

[56]  Les A. Piegl,et al.  Interactive Data Interpolation by Rational Bezier Curves , 1987, IEEE Computer Graphics and Applications.

[57]  Parris K. Egbert,et al.  A Gestural Interface to Free-Form Deformation , 2003, Graphics Interface.

[58]  Hyewon Seo,et al.  Free-Form Sketching of Self-Occluding Objects , 2007, IEEE Computer Graphics and Applications.

[59]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  Xavier Granier,et al.  Improved skeleton extraction and surface generation for sketch-based modeling , 2007, GI '07.

[61]  David Salesin,et al.  A sketching interface for articulated figure animation , 2006, SIGGRAPH 2006.

[62]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[63]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[64]  Takeo Igarashi,et al.  Floral diagrams and inflorescences: interactive flower modeling using botanical structural constraints , 2007, ACM Trans. Graph..

[65]  Hiromasa Suzuki,et al.  A two-stage approach for interpreting line drawings of curved objects , 2004, SBM'04.

[66]  Takeo Igarashi,et al.  Plushie: an interactive design system for plush toys , 2007, ACM Trans. Graph..

[67]  John F. Hughes,et al.  SKETCH: An Interface for Sketching 3D Scenes , 1996, SIGGRAPH.

[68]  Pedro Santos,et al.  Constraint stroke-based oversketching for 3D curves , 2004, SBM'04.

[69]  Elaine Cohen,et al.  Real time spline curves from interactively sketched data , 1990, I3D '90.

[70]  Gershon Elber,et al.  Inferring 3D models from freehand sketches and constraints , 1997, Comput. Aided Des..

[71]  John F. Hughes,et al.  SmoothSketch: 3D free-form shapes from complex sketches , 2006, SIGGRAPH '06.

[72]  Siti Rokhmah Mohd Shukri,et al.  A Sketch-and-Spray Interface for Modeling Trees , 2007, Smart Graphics.

[73]  Thomas F. Stahovich,et al.  Sketch based interfaces: early processing for sketch understanding , 2001, PUI '01.

[74]  Joseph J. LaViola,et al.  MathPad2: a system for the creation and exploration of mathematical sketches , 2004, ACM Trans. Graph..

[75]  Kenneth Rose,et al.  Eurographics Symposium on Geometry Processing (2007) Developable Surfaces from Arbitrary Sketched Boundaries , 2022 .

[76]  Randall Davis,et al.  Scale-space based feature point detection for digital ink , 2007, SIGGRAPH '07.

[77]  Brian Wyvill,et al.  Sketch Based Construction and Rendering of Implicit Models , 2005, CAe.

[78]  Christine Alvarado,et al.  SketchREAD: a multi-domain sketch recognition engine , 2004, UIST '04.

[79]  Ligang Liu,et al.  Easy Mesh Cutting , 2006, Comput. Graph. Forum.

[80]  Jon Trinder,et al.  The Humane Interface: New Directions for Designing Interactive Systems , 2002, Interact. Learn. Environ..

[81]  Eric and Gross Mark Schweikhardt.  Digital Clay: Deriving Digital Models from Freehand Sketches , 1998 .

[82]  Meenakshisundaram Gopi,et al.  Sketching Free-form Surfaces Using Network of Curves , 2005, SBM.

[83]  Özgür Ulusoy,et al.  KiMPA: A Kinematics-Based Method for Polygon Approximation , 2002, ADVIS.

[84]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[85]  David G. Ullman,et al.  The design capture system : capturing back-of-the-envelope sketches , 1990 .

[86]  Levent Burak Kara,et al.  An image-based, trainable symbol recognizer for hand-drawn sketches , 2005, Comput. Graph..

[87]  Mario Costa Sousa,et al.  Sketch-Based Mesh Augmentation , 2005, SBM.

[88]  Ralph R. Martin,et al.  Frontal geometry from sketches of engineering objects: is line labelling necessary? , 2005, Comput. Aided Des..

[89]  K. C. Hui,et al.  Generating subdivision surfaces from profile curves , 2007, Comput. Aided Des..

[90]  Takeo Igarashi,et al.  A suggestive interface for 3D drawing , 2001, SIGGRAPH Courses.

[91]  Manuel J. Fonseca,et al.  Sketching User Interfaces with Visual Patterns , .

[92]  Takeo Igarashi,et al.  Clothing manipulation , 2003, ACM Trans. Graph..

[93]  Levent Burak Kara,et al.  An efficient graph-based symbol recognizer , 2006, SBM'06.

[94]  David Pugh,et al.  Designing solid objects using interactive sketch interpretation , 1992, I3D '92.

[95]  Venu Govindaraju,et al.  Document image analysis: A primer , 2002 .

[96]  Marc Alexa,et al.  A sketch-based interface for detail-preserving mesh editing , 2005, SIGGRAPH 2005.

[97]  Loïc Barthe,et al.  Interactive modelling from sketches using spherical implicit functions , 2004, AFRIGRAPH '04.

[98]  Dean Rubine,et al.  Specifying gestures by example , 1991, SIGGRAPH.

[99]  M. Gross,et al.  Digital clay: deriving digital models from freehand sketches , 2000 .