SketchingWithHands: 3D Sketching Handheld Products with First-Person Hand Posture

We present SketchingWithHands, a 3D sketching system that incorporates a hand-tracking sensor. The system enables product designers to easily capture desired hand postures from a first-person point of view at any time and to use the captured hand information to explore handheld product concepts by 3D sketching while keeping the proper scale and usage of the products. Based on the analysis of design practices and drawing skills in the art and design literature, we suggest novel ideas for efficiently acquiring hand postures (palm-pinning widget, front and center mirrors, responsive spangles), for quickly creating and easily adjusting sketch planes (modified tick-triggered, orientable and shiftable sketch planes), for appropriately starting 3D sketching products with hand information (hand skeleton, grip axis), and for practically increasing user throughput (intensifier, rough and precise erasers)---all of which are coherently and consistently integrated in our system. A user test by ten industrial design students and an in-depth discussion show that our system is both useful and usable in designing handheld products.

[1]  Jack Hamm,et al.  Drawing the Head and Figure , 1982 .

[2]  Y. Guiard.  Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. , 1987, Journal of motor behavior.

[3]  Andrew Roberts,et al.  3-Draw: a tool for designing 3D shapes , 1991, IEEE Computer Graphics and Applications.

[4]  R. W. Mitchell,et al.  Kinesthetic-visual matching and the self-concept as explanations of mirror-self-recognition. , 1997 .

[5]  Alex Pentland,et al.  The ALIVE system: wireless, full-body interaction with autonomous agents , 1997, Multimedia Systems.

[6]  Takanori Maesako,et al.  HyperMirror: toward pleasant-to-use video mediated communication system , 1998, CSCW '98.

[7]  Peter Schröder,et al.  Surface drawing: creating organic 3D shapes with the hand and tangible tools , 2001, CHI.

[8]  Ying Wu,et al.  Hand modeling, analysis and recognition , 2001, IEEE Signal Process. Mag..

[9]  Karan Singh,et al.  Eurographics/siggraph Symposium on Computer Animation (2003) Handrix: Animating the Human Hand , 2003 .

[10]  Abhishek Ranjan,et al.  A suggestive interface for image guided 3D sketching , 2004, CHI.

[11]  Ramesh Raskar,et al.  Epipolar methods for multi-view sketching , 2004, SBM'04.

[12]  Kiia Kallio.  3D6B Editor: Projective 3D Sketching with Line-Based Rendering , 2005, SBM.

[13]  Yang Li,et al.  Experimental analysis of mode switching techniques in pen-based user interfaces , 2005, CHI.

[14]  Ehud Sharlin,et al.  The 3D Tractus: a three-dimensional drawing board , 2006, First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP '06).

[15]  Kurt Hanks,et al.  Rapid Viz, Third Edition: A New Method for the Rapid Visualitzation of Ideas , 2006 .

[16]  Holly E. Rushmeier,et al.  The Mental Canvas: A Tool for Conceptual Architectural Design and Analysis , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[17]  Marie-Paule Cani,et al.  HandNavigator: hands-on interaction for desktop virtual reality , 2008, VRST '08.

[18]  Ehud Sharlin,et al.  Napkin sketch: handheld mixed reality 3D sketching , 2008, VRST '08.

[19]  Ravin Balakrishnan,et al.  ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models , 2008, UIST '08.

[20]  Ryan Schmidt,et al.  Analytic drawing of 3D scaffolds , 2009, ACM Trans. Graph..

[21]  Mario Costa Sousa,et al.  Sketch-based modeling: A survey , 2009, Comput. Graph..

[22]  Koos Eissen,et al.  Sketching: Drawing Techniques for Product Designers , 2009 .

[23]  Ravin Balakrishnan,et al.  EverybodyLovesSketch: 3D sketching for a broader audience , 2009, UIST '09.

[24]  Scott Robertson.  Lift Off: Air Vehicle Sketches & Renderings from the Drawthrough Collection , 2010 .

[25]  Koos Eissen,et al.  Sketching: The Basics , 2011 .

[26]  Holly E. Rushmeier,et al.  Insitu: sketching architectural designs in context , 2011, ACM Trans. Graph..

[27]  Sylvain Paris,et al.  6D hands: markerless hand-tracking for computer aided design , 2011, UIST.

[28]  CrossShade: shading concept sketches using cross-section curves , 2012, ACM Trans. Graph..

[29]  Andrew Wilson,et al.  MirageTable: freehand interaction on a projected augmented reality tabletop , 2012, CHI.

[30]  Levent Burak Kara,et al.  Conceptual design and modification of freeform surfaces using dual shape representations in augmented reality environments , 2012, Comput. Aided Des..

[31]  C. Karen Liu,et al.  Synthesis of detailed hand manipulations using contact sampling , 2012, ACM Trans. Graph..

[32]  Kevin Henry.  Drawing for Product Designers , 2012 .

[33]  Joaquim A. Jorge,et al.  Mockup builder: direct 3D modeling on and above the surface in a continuous interaction space , 2012, Graphics Interface.

[34]  Xiang Cao,et al.  BodyAvatar: creating freeform 3D avatars using first-person body gestures , 2013, UIST.

[35]  Tovi Grossman,et al.  YouMove: enhancing movement training with an augmented reality mirror , 2013, UIST.

[36]  Adrien Bousseau,et al.  True2Form: 3D curve networks from 2D sketches via selective regularization , 2014, ACM Trans. Graph..

[37]  Ismail Khalid Kazmi,et al.  A Survey of Sketch Based Modeling Systems , 2014, 2014 11th International Conference on Computer Graphics, Imaging and Visualization.

[38]  M. Nowicki,et al.  FOR LEAP MOTION CONTROLLER , 2014 .

[39]  Andrew W. Fitzgibbon,et al.  Accurate, Robust, and Flexible Real-time Hand Tracking , 2015, CHI.

[40]  Madeline Gannon,et al.  Tactum: A Skin-Centric Approach to Digital Design and Fabrication , 2015, CHI.

[41]  Maud Marchal,et al.  THING: Introducing a Tablet-based Interaction Technique for Controlling 3D Hand Models , 2015, CHI.

[42]  Karan Singh,et al.  SecondSkin: sketch-based construction of layered 3D models , 2015, ACM Trans. Graph..