On the Efficiency of a VR Hand Gesture-Based Interface for 3D Object Manipulations in Conceptual Design

ABSTRACT In the early stages of 3D design, sketches are used to quickly conceptualize ideas and gain insight into problems and possible solutions. Computer-aided design tools are widely used for 3D modeling and design, but their required precision and 2D mouse and screen-based interface inhibit the flow of ideas. A study was conducted to explore the efficiency of hand tracking and virtual reality (VR) for 3D object manipulations in conceptual design. Based on existing research on conceptual design and hand gestures, an intuitive hand-based interaction model is proposed. An experiment on basic 3D manipulation shows that participants using a simple VR and hand-tracking interface prototype have similar performance to those using a traditional mouse and screen interface. For the improvement of gestural conceptual design interfaces, the relevant issues are identified.

[1]  Frank Bretz,et al.  Simultaneous Inference , 2020, Biostatistics Decoded.

[2]  Andy Cockburn,et al.  FingARtips: gesture based direct manipulation in Augmented Reality , 2004, GRAPHITE '04.

[3]  Steven K. Feiner,et al.  Cross-dimensional gestural interaction techniques for hybrid immersive environments , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[4]  Raffaele de Amicis,et al.  Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design , 2002, ISMAR.

[5]  Manuel Contero,et al.  Direct Modeling: from Sketches to 3D Models , 2002 .

[6]  Vesna Popovic,et al.  Intuitive Use of Products , 2002 .

[7]  Timothy Marsh,et al.  Shape your imagination: iconic gestural-based interaction , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[8]  Ipke Wachsmuth,et al.  Interpretation of Shape-Related Iconic Gestures in Virtual Environments , 2001, Gesture Workshop.

[9]  Satoshi Matsuoka,et al.  Teddy: A Sketching Interface for 3D Freeform Design , 1999, SIGGRAPH Courses.

[10]  John F. Hughes,et al.  SKETCH: An Interface for Sketching 3D Scenes , 1996, SIGGRAPH.

[11]  Stefan Kopp,et al.  Gesture-based Object Recognition using Histograms of Guiding Strokes , 2012, BMVC.

[12]  Jane Doherty,et al.  The effects of sign characteristics on sign acquisition and retention: an integrative review of the literature , 1985 .

[13]  Keith Duncan,et al.  Cognitive Engineering , 2017, Encyclopedia of GIS.

[14]  Weiming Shen,et al.  Collaborative conceptual design - state of the art and future trends , 2002, Comput. Aided Des..

[15]  Daniel E. Whitney,et al.  Concurrent Design of Products and Processes: A Strategy for the Next Generation in Manufacturing , 1989 .

[16]  Martha W. Alibali,et al.  Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information , 2005, Spatial Cogn. Comput..

[17]  Jason Jerald,et al.  MakeVR: A 3D world-building interface , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[18]  Fons J. Verbeek,et al.  Pointing Task Evaluation of Leap Motion Controller in 3D Virtual Environment , 2014 .

[19]  R. Ellis,et al.  Micro-affordance: the potentiation of components of action by seen objects. , 2000, British journal of psychology.

[20]  Monica Bordegoni.  Parallel use of hand gestures and force-input device for interacting with 3d and virtual reality environments , 1994, Int. J. Hum. Comput. Interact..

[21]  Christoph W. Borst,et al.  A Spring Model for Whole-Hand Virtual Grasping , 2006, Presence: Teleoperators & Virtual Environments.

[22]  Lyle L. Lloyd,et al.  Manual Sign Translucency and Referential Concreteness in the Learning of Signs , 2013 .

[23]  Eric Jamet,et al.  Effects of Stereoscopic Display on Learning and User Experience in an Educational Virtual Environment , 2017, Int. J. Hum. Comput. Interact..

[24]  Joaquim A. Jorge,et al.  Towards Calligraphic Interfaces: Sketching 3D Scenes with Gestures and Context Icons , 2000, WSCG.

[25]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[26]  Steven M. Smith,et al.  Metrics for measuring ideation effectiveness , 2003 .

[27]  Wolfgang Stuerzlinger,et al.  Group Selection Techniques for Efficient 3D Modeling , 2006, 3D User Interfaces (3DUI'06).

[28]  T. Hothorn,et al.  Simultaneous Inference in General Parametric Models , 2008, Biometrical journal. Biometrische Zeitschrift.

[29]  Constantine Stephanidis,et al.  Universal Access in Human-Computer Interaction. Access to Interaction , 2015, Lecture Notes in Computer Science.

[30]  Soh-Khim Ong,et al.  Vision-Based Hand Interaction in Augmented Reality Environment , 2011, Int. J. Hum. Comput. Interact..

[31]  Rahinah Ibrahim,et al.  Impacts of VR 3D sketching on novice designers’ spatial cognition in collaborative conceptual architectural design , 2011 .

[32]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[33]  J. D. Ruiter.  The production of gesture and speech , 2000 .

[34]  Jibo He,et al.  A Comparison of Gaze-Based and Gesture-Based Input for a Point-and-Click Task , 2015, HCI.

[35]  Oliver Bimber,et al.  A Translucent Sketchpad for the Virtual Table Exploring Motion‐based Gesture Recognition , 1999, Comput. Graph. Forum.

[36]  R Wesp,et al.  Gestures maintain spatial imagery. , 2001, The American journal of psychology.

[37]  Philip Cash,et al.  Prototyping with your hands: the many roles of gesture in the communication of design concepts , 2016 .

[38]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[39]  Casper G.C. van Dijk,et al.  New insights in computer-aided conceptual design , 1995 .

[40]  Martha W. Alibali,et al.  On the Tip of the Mind: Gesture as a Key to Conceptualization , 2004 .

[41]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[42]  Rashid Ansari,et al.  Multimodal human discourse: gesture and speech , 2002, TCHI.

[43]  David Zeltzer,et al.  A design method for “whole-hand” human-computer interaction , 1993, TOIS.

[44]  R. Gibbs.  Embodiment and Cognitive Science: Concepts , 2005 .

[45]  Frances M. T. Brazier,et al.  On the Usability and Effectiveness of Different Interaction Types in Augmented Reality , 2015, Int. J. Hum. Comput. Interact..

[46]  Ezequiel Morsella,et al.  The role of gestures in spatial working memory and speech. , 2004, The American journal of psychology.

[47]  Michael F. Deering,et al.  HoloSketch: a virtual reality sketching/animation tool , 1995, TCHI.