Window-Shaping: 3D Design Ideation by Creating on, Borrowing from, and Looking at the Physical World

We present, Window-Shaping, a tangible mixed-reality (MR) interaction metaphor for design ideation that allows for the direct creation of 3D shapes on and around physical objects. Using the sketch-and-inflate scheme, our metaphor enables quick design of dimensionally consistent and visually coherent 3D models by borrowing visual and dimensional attributes from existing physical objects without the need for 3D reconstruction or fiducial markers. Through a preliminary evaluation of our prototype application we demonstrate the expressiveness provided by our design workflow, the effectiveness of our interaction scheme, and the potential of our metaphor.

[1]  Madeline Gannon,et al.  Tactum: A Skin-Centric Approach to Digital Design and Fabrication , 2015, CHI.

[2]  Peter Dalsgård,et al.  The productive role of material design artefacts in participatory design events , 2012, NordiCHI.

[3]  Ken Perlin,et al.  T(ether): spatially-aware handhelds, gestures and proprioception for multi-user 3D modeling and animation , 2014, SUI.

[4]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[5]  Steve Harrison,et al.  Investigating the relationship between imagery and rationale in design , 2010, Conference on Designing Interactive Systems.

[6]  Andrew Roberts,et al.  3-Draw: a tool for designing 3D shapes , 1991, IEEE Computer Graphics and Applications.

[7]  Tobias Höllerer,et al.  Interpreting 2D gesture annotations in 3D augmented reality , 2016, 2016 IEEE Symposium on 3D User Interfaces (3DUI).

[8]  Hod Lipson,et al.  ModelCraft: capturing freehand annotations and edits on physical 3D models , 2006, UIST.

[9]  Niloy J. Mitra,et al.  SmartCanvas: Context‐inferred Interpretation of Sketches for Preparatory Design Studies , 2016, Comput. Graph. Forum.

[10]  Austin S. Lee,et al.  Second surface: multi-user spatial collaboration system based on augmented reality , 2012, SIGGRAPH Asia Emerging Technologies.

[11]  David A. Forsyth,et al.  BeThere: 3D mobile collaboration with spatial input , 2013, CHI.

[12]  Caroline Hummels,et al.  The reflective transformative design process , 2009, CHI Extended Abstracts.

[13]  Eyal Ofek,et al.  SnapToReality: Aligning Augmented Reality to the Real World , 2016, CHI.

[14]  Ravin Balakrishnan,et al.  Elasticurves: exploiting stroke dynamics and inertia for the real-time neatening of sketched 2D curves , 2011, UIST '11.

[15]  Kenny Mitchell,et al.  Augmented creativity: bridging the real and virtual worlds to enhance creative play , 2015, SIGGRAPH Asia Mobile Graphics and Interactive Applications.

[16]  Shumin Zhai,et al.  SHARK2: a large vocabulary shorthand writing system for pen-based computers , 2004, UIST '04.

[17]  Daniel Cohen-Or,et al.  3-Sweep , 2013, ACM Trans. Graph..

[18]  Philip H. S. Torr,et al.  VideoTrace: rapid interactive scene modelling from video , 2007, ACM Trans. Graph..

[19]  Ehud Sharlin,et al.  Napkin sketch: handheld mixed reality 3D sketching , 2008, VRST '08.

[20]  Marc Alexa,et al.  FiberMesh: designing freeform surfaces with 3D curves , 2007, SIGGRAPH 2007.

[21]  Borut Zalik,et al.  Sweep‐line algorithm for constrained Delaunay triangulation , 2008, Int. J. Geogr. Inf. Sci..

[22]  Brian P. Bailey,et al.  Getting inspired!: understanding how and why examples are used in creative design practice , 2009, CHI.

[23]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[24]  David Kim,et al.  MixFab: a mixed-reality environment for personal fabrication , 2014, CHI.

[25]  Peter Schröder,et al.  Surface drawing: creating organic 3D shapes with the hand and tangible tools , 2001, CHI.

[26]  Thomas Baudel,et al.  A mark-based interaction paradigm for free-hand drawing , 1994, UIST '94.

[27]  Takeo Igarashi,et al.  Situated modeling: a shape-stamping interface with tangible primitives , 2012, Tangible and Embedded Interaction.

[28]  Karthik Ramani,et al.  Tracing and sketching performance using blunt-tipped styli on direct-touch tablets , 2014, AVI.

[29]  Fumihisa Shibata,et al.  Virtual Handcrafting: Building Virtual Wood Models Using ToolDevice , 2014, Proceedings of the IEEE.

[30]  Raffaele de Amicis,et al.  Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design , 2002, ISMAR.

[31]  Karthik Ramani,et al.  MobiSweep: Exploring Spatial Design Ideation Using a Smartphone as a Hand-held Reference Plane , 2016, Tangible and Embedded Interaction.

[32]  Joseph J. LaViola,et al.  CavePainting: a fully immersive 3D artistic medium and interactive experience , 2001, I3D '01.

[33]  Takeo Igarashi,et al.  Modeling-in-Context: User Design of Complementary Objects with a Single Photo , 2010, SBIM.

[34]  Yunbo Zhang,et al.  RealFusion: An Interactive Workflow for Repurposing Real-World Objects towards Early-stage Creative Ideation , 2016, Graphics Interface.

[35]  Joaquim A. Jorge,et al.  NaturaSketch: Modeling from Images and Natural Sketches , 2011, IEEE Computer Graphics and Applications.

[36]  Lin Wang,et al.  Garment modeling with a depth camera , 2015, ACM Trans. Graph..

[37]  Holly E. Rushmeier,et al.  Insitu: sketching architectural designs in context , 2011, ACM Trans. Graph..