Mobi3DSketch: 3D Sketching in Mobile AR

Mid-air 3D sketching has been mainly explored in Virtual Reality (VR) and typically requires special hardware for motion capture and immersive, stereoscopic displays. The recently developed motion tracking algorithms allow real-time tracking of mobile devices, and have enabled a few mobile applications for 3D sketching in Augmented Reality (AR). However, they are more suitable for making simple drawings only, since they do not consider special challenges with mobile AR 3D sketching, including the lack of stereo display, narrow field of view, and the coupling of 2D input, 3D input and display. To address these issues, we present Mobi3DSketch, which integrates multiple sources of inputs with tools, mainly different versions of 3D snapping and planar/curves surface proxies. Our multimodal interface supports both absolute and relative drawing, allowing easy creation of 3D concept designs in situ. The effectiveness and expressiveness of Mobi3DSketch are demonstrated via a pilot study.

[1]  Eyal Ofek,et al.  SnapToReality: Aligning Augmented Reality to the Real World , 2016, CHI.

[2]  Eric A. Bier.  Snap-dragging in three dimensions , 1990, I3D '90.

[3]  Ehud Sharlin,et al.  Napkin sketch: handheld mixed reality 3D sketching , 2008, VRST '08.

[4]  Shun'ichi Tano,et al.  Truly Useful 3D Drawing System for Professional Designer by "Life-Sized and Operable" Feature and New Interaction , 2013, INTERACT.

[5]  William Buxton,et al.  Creating principal 3D curves with digital tape drawing , 2002, CHI.

[6]  Andrew Y. C. Nee,et al.  Augmented reality applications in design and manufacturing , 2012 .

[7]  Paul Asente,et al.  Advanced drawing beautification with ShipShape , 2016, Comput. Graph..

[8]  Changqing Zou,et al.  Progressive 3D Reconstruction of Planar-Faced Manifold Objects with DRF-Based Line Drawing Decomposition. , 2015, IEEE transactions on visualization and computer graphics.

[9]  Ke Huo,et al.  Window-Shaping: 3D Design Ideation by Creating on, Borrowing from, and Looking at the Physical World , 2017, TEI.

[10]  Cindy Grimm,et al.  Just DrawIt: a 3D sketching system , 2012, SBIM '12.

[11]  Peter Schröder,et al.  Surface drawing: creating organic 3D shapes with the hand and tangible tools , 2001, CHI.

[12]  Niloy J. Mitra,et al.  SmartCanvas: Context‐inferred Interpretation of Sketches for Preparatory Design Studies , 2016, Comput. Graph. Forum.

[13]  Jihad El-Sana,et al.  In-place 3D sketching for authoring and augmenting mechanical systems , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[14]  Brandon Yee,et al.  Augmented Reality In-Situ 3D Sketching of Physical Objects , 2009 .

[15]  Adrien Bousseau,et al.  True2Form: 3D curve networks from 2D sketches via selective regularization , 2014, ACM Trans. Graph..

[16]  Ryan Schmidt,et al.  Analytic drawing of 3D scaffolds , 2009, ACM Trans. Graph..

[17]  Yongkwan Kim,et al.  Agile 3D Sketching with Air Scaffolding , 2018, CHI.

[18]  Joseph J. LaViola,et al.  CavePainting: a fully immersive 3D artistic medium and interactive experience , 2001, I3D '01.

[19]  Wendy E. Mackay,et al.  Beyond Snapping: Persistent, Tweakable Alignment and Distribution with StickyLines , 2016, UIST.

[20]  Michael F. Deering,et al.  HoloSketch: a virtual reality sketching/animation tool , 1995, TCHI.

[21]  George Drettakis,et al.  Drawing for Illustration and Annotation in 3D , 2001, Comput. Graph. Forum.

[22]  Yvonne Rogers,et al.  Fat Finger Worries: How Older and Younger Users Physically Interact with PDAs , 2005, INTERACT.

[23]  David H. Laidlaw,et al.  Drawing on Air: Input Techniques for Controlled 3D Line Illustration , 2007, IEEE Transactions on Visualization and Computer Graphics.

[24]  Julie Dorsey,et al.  Insitu: sketching architectural designs in context , 2011, SA '11.

[25]  Rubaiat Habib Kazi,et al.  SymbiosisSketch: Combining 2D & 3D Sketching for Designing Detailed 3D Objects in Situ , 2018, CHI.

[26]  Yongkwan Kim,et al.  SketchingWithHands: 3D Sketching Handheld Products with First-Person Hand Posture , 2016, UIST.

[27]  Pengfei Xu,et al.  SweepCanvas: Sketch-based 3D Prototyping on an RGB-D Image , 2017, UIST.

[28]  Tomoyuki Nishita,et al.  Towards Digital Refocusing from a Single Photograph , 2007 .

[29]  Hod Lipson,et al.  Optimization-based reconstruction of a 3D object from a single freehand line drawing , 1996, Comput. Aided Des..

[30]  Anastasios I. Mourikis,et al.  High-precision, consistent EKF-based visual-inertial odometry , 2013, Int. J. Robotics Res..

[31]  Ryan Schmidt,et al.  Analytic drawing of 3D scaffolds , 2009, SIGGRAPH 2009.

[32]  Charlie C. L. Wang.  Drawing on Air: Input Techniques for Controlled 3D Line Illustration , 2007 .

[33]  Yunbo Zhang,et al.  RealFusion: An Interactive Workflow for Repurposing Real-World Objects towards Early-stage Creative Ideation , 2016, Graphics Interface.

[34]  Cindy Grimm,et al.  Just DrawIt : a 3 D sketching system , 2012 .

[35]  Pengfei Xu,et al.  Model-Guided 3D Sketching , 2019, IEEE Transactions on Visualization and Computer Graphics.

[36]  Ravin Balakrishnan,et al.  EverybodyLovesSketch: 3D sketching for a broader audience , 2009, UIST '09.

[37]  Rubaiat Habib Kazi,et al.  Experimental Evaluation of Sketching on Surfaces in VR , 2017, CHI.

[38]  Andrew Roberts,et al.  3-Draw: a tool for designing 3D shapes , 1991, IEEE Computer Graphics and Applications.

[39]  Holly E. Rushmeier,et al.  The Mental Canvas: A Tool for Conceptual Architectural Design and Analysis , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[40]  Karthik Ramani,et al.  MobiSweep: Exploring Spatial Design Ideation Using a Smartphone as a Hand-held Reference Plane , 2016, Tangible and Embedded Interaction.

[41]  Daniel F. Keefe,et al.  Lift-Off: Using Reference Imagery and Freehand Sketching to Create 3D Models in VR , 2016, IEEE Transactions on Visualization and Computer Graphics.

[42]  Michael Hoffmann,et al.  Hyve-3D and the 3D Cursor: Architectural co-design with freedom in Virtual Reality , 2016 .

[43]  Kiia Kallio.  3D6B Editor: Projective 3D Sketching with Line-Based Rendering , 2005, SBM.

[44]  Ken Perlin,et al.  T(ether): spatially-aware handhelds, gestures and proprioception for multi-user 3D modeling and animation , 2014, SUI.