Dynamic furniture modeling through assembly instructions

We present a technique for parsing widely used furniture assembly instructions, and reconstructing the 3D models of furniture components and their dynamic assembly process. Our technique takes as input a multi-step assembly instruction in a vector graphic format and starts to group the vector graphic primitives into semantic elements representing individual furniture parts, mechanical connectors (e.g., screws, bolts and hinges), arrows, visual highlights, and numbers. To reconstruct the dynamic assembly process depicted over multiple steps, our system identifies previously built 3D furniture components when parsing a new step, and uses them to address the challenge of occlusions while generating new 3D components incrementally. With a wide range of examples covering a variety of furniture types, we demonstrate the use of our system to animate the 3D furniture assembly process and, beyond that, the semantic-aware furniture editing as well as the fabrication of personalized furnitures.

[1]  Luc Van Gool,et al.  Real time head pose estimation with random regression forests , 2011, CVPR 2011.

[2]  Marc Alexa,et al.  How do humans sketch objects? , 2012, ACM Trans. Graph..

[3]  Paul U. Lee,et al.  Lines, Blobs, Crosses and Arrows: Diagrammatic Communication with Schematic Figures , 2000, Diagrams.

[4]  Wojciech Matusik,et al.  Design and fabrication by example , 2014, ACM Trans. Graph..

[5]  Hao Zhang,et al.  Foldabilizing furniture , 2015, ACM Trans. Graph..

[6]  ZhengChangxi,et al.  Dynamic furniture modeling through assembly instructions , 2016 .

[7]  Karan Singh,et al.  SecondSkin: sketch-based construction of layered 3D models , 2015, ACM Trans. Graph..

[8]  Roberto Cipolla,et al.  Semantic texton forests for image categorization and segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Karl Tombre,et al.  Graphics Recognition Algorithms and Systems , 1997, Lecture Notes in Computer Science.

[10]  Robert M. Haralick,et al.  Understanding engineering drawings , 1982, Comput. Graph. Image Process..

[11]  Randall Davis,et al.  ChemInk: a natural real-time recognition system for chemical drawings , 2011, IUI '11.

[12]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Robert M. Haralick,et al.  Understanding engineering drawings , 1982, Comput. Graph. Image Process..

[14]  Takeo Igarashi,et al.  SketchChair: an all-in-one chair design system for end users , 2011, Tangible and Embedded Interaction.

[15]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Karl Tombre,et al.  Graphics recognition : algorithms and systems : Second International Workshop, GREC ʾ97, Nancy, France, August 22-23, 1997 : selected papers , 1998 .

[17]  Ligang Liu,et al.  Animated construction of line drawings , 2011, ACM Trans. Graph..

[18]  Pat Hanrahan,et al.  Designing effective step-by-step assembly instructions , 2003, ACM Trans. Graph..

[19]  Yin Yang,et al.  An interactive approach for functional prototype recovery from a single RGBD image , 2016, Computational Visual Media.

[20]  Adrien Bousseau,et al.  True2Form: 3D curve networks from 2D sketches via selective regularization , 2014, ACM Trans. Graph..

[21]  Kun Zhou,et al.  Interpreting concept sketches , 2013, ACM Trans. Graph..

[22]  Rynson W. H. Lau,et al.  Data-driven segmentation and labeling of freehand sketches , 2014, ACM Trans. Graph..

[23]  Takeo Igarashi,et al.  Converting 3D furniture models to fabricatable parts and connectors , 2011, ACM Trans. Graph..

[24]  Takeo Igarashi,et al.  Structured annotations for 2D-to-3D modeling , 2009, ACM Trans. Graph..

[25]  Kun Zhou,et al.  Discriminative Sketch‐based 3D Model Retrieval via Robust Shape Matching , 2011, Comput. Graph. Forum.

[26]  Ryan Schmidt,et al.  Analytic drawing of 3D scaffolds , 2009, ACM Trans. Graph..

[27]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[28]  Dieter Schmalstieg,et al.  Retargeting Technical Documentation to Augmented Reality , 2015, CHI.

[29]  Werner Hartmann,et al.  Authoring of a mixed reality furniture assembly instructor , 2003, SIGGRAPH '03.

[30]  Takeo Igarashi,et al.  Guided exploration of physically valid shapes for furniture design , 2012, ACM Trans. Graph..

[31]  Daniel Cohen-Or,et al.  3-Sweep , 2013, ACM Trans. Graph..

[32]  Daniel Cohen-Or,et al.  Geosemantic Snapping for Sketch‐Based Modeling , 2013, Comput. Graph. Forum.

[33]  Wilmot Li,et al.  Creating works-like prototypes of mechanical objects , 2014, ACM Trans. Graph..

[34]  Harry Shum,et al.  Sketching reality: Realistic interpretation of architectural designs , 2008, TOGS.

[35]  Liqing Zhang,et al.  Free Hand-Drawn Sketch Segmentation , 2012, ECCV.

[36]  Ravin Balakrishnan,et al.  ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models , 2008, UIST '08.

[37]  Shi-Min Hu,et al.  Sketch2Scene: sketch-based co-retrieval and co-placement of 3D models , 2013, ACM Trans. Graph..

[38]  Juan B. Mena,et al.  State of the art on automatic road extraction for GIS update: a novel classification , 2003, Pattern Recognit. Lett..

[39]  Joseph J. LaViola,et al.  MathPad2: a system for the creation and exploration of mathematical sketches , 2004, SIGGRAPH 2004.

[40]  Pat Hanrahan,et al.  Identification and validation of cognitive design principles for automated generation of assembly instructions , 2004, AVI.

[41]  Eitan Grinspun,et al.  Parsing sewing patterns into 3D garments , 2013, ACM Trans. Graph..

[42]  Joseph LaViola Session details: An introduction to sketch-based interfaces , 2006, SIGGRAPH 2006.

[43]  Wilmot Li,et al.  Illustrating how mechanical assemblies work , 2010, CACM.

[44]  Barbara Tversky,et al.  Animation: can it facilitate? , 2002, Int. J. Hum. Comput. Stud..

[45]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[46]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  David Salesin,et al.  Interactive Image-Based Exploded View Diagrams , 2004, Graphics Interface.

[48]  David P. Dobkin,et al.  A search engine for 3D models , 2003, TOGS.

[49]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[50]  Kun Zhou,et al.  Interactive images , 2012, ACM Trans. Graph..

[51]  John F. Hughes,et al.  SKETCH: An Interface for Sketching 3D Scenes , 1996, SIGGRAPH.

[52]  Marc Alexa,et al.  Sketch-based shape retrieval , 2012, ACM Trans. Graph..

[53]  Shi-Min Hu,et al.  Interactive Image‐Guided Modeling of Extruded Shapes , 2014, Comput. Graph. Forum.

[54]  Dieter Fox,et al.  DuploTrack: a real-time system for authoring and guiding duplo block assembly , 2012, UIST.

[55]  Levent Burak Kara,et al.  Combining geometry and domain knowledge to interpret hand-drawn diagrams , 2005, Comput. Graph..

[56]  John F. Hughes,et al.  SmoothSketch: 3D free-form shapes from complex sketches , 2006, SIGGRAPH '06.