User-defined walking-in-place gestures for VR locomotion

Abstract Locomotion is one of the fundamental interactions in VR. As a relatively easy and simple method to implement VR locomotion, walking-in-place (WIP) techniques have been actively developed, which have shown advantages in terms of spatial constraints and immersion compared to real walking or controller-based interaction. However, existing WIP gestures have been largely adapted or designed from the perspective of developers, not the users, which may result in not only higher cognitive load to learn and memorize, but also worse presence and increased sensory conflict in VR. Therefore, this study aims to elicit and evaluate WIP gestures for different walking directions (forward, sideways, and backward) from users so that a complete user-defined WIP gesture set for VR locomotion can be generated. Two sequential user studies were conducted. In Experiment 1, 20 participants experienced the movement while wearing the VR headset and elicited the gesture for each of 8 walking directions. The grouping analysis revealed that Turn body + Stepping-in-place (SIP) and Step one foot + SIP/Rock/Stay were promising WIP gesture sets for VR locomotion. In the follow-up experiment (Experiment 2), 21 new participants experienced and compared the generated gesture sets and existing ones. The results showed that SIP performed the best for forward walking, while Step one foot + SIP/Rock/Stay were promising for other walking directions, which depended on application scenarios and the tradeoff between efficiency and naturalness. The generated WIP gesture sets from users can be used in VR applications to provide a better user experience and greater movement options in VR locomotion.

[1]  Joel Lanir,et al.  Better Understanding of Foot Gestures: An Elicitation Study , 2018, CHI.

[2]  Maud Marchal,et al.  Joyman: A human-scale joystick for navigating in virtual worlds , 2011, 2011 IEEE Symposium on 3D User Interfaces (3DUI).

[3]  Eelke Folmer,et al.  VR-STEP: Walking-in-Place using Inertial Sensing for Hands Free Navigation in Mobile VR Environments , 2016, CHI.

[4]  Natacha Métayer,et al.  Literature review of locomotion techniques in virtual reality , 2020, Int. J. Virtual Real..

[5]  Huiyue Wu,et al.  Understanding freehand gestures: a study of freehand gestural interaction for immersive VR shopping applications , 2019, Human-centric Computing and Information Sciences.

[6]  Eelke Folmer,et al.  Virtual Locomotion: A Survey , 2020, IEEE Transactions on Visualization and Computer Graphics.

[7]  Majed Al Zayer,et al.  Handsfree Omnidirectional VR Navigation using Head Tilt , 2017, CHI.

[8]  Huiyue Wu,et al.  The Gesture Disagreement Problem in Free-hand Gesture Interaction , 2018, Int. J. Hum. Comput. Interact..

[9]  Bernhard Jenny,et al.  Elicitation study investigating hand and foot gesture interaction for immersive maps in augmented reality , 2020, Cartography and Geographic Information Science.

[10]  Robert W. Lindeman,et al.  On Your Feet!: Enhancing Vection in Leaning-Based Interfaces through Multisensory Stimuli , 2016, SUI.

[11]  Luca Chittaro,et al.  Locomotion in Place in Virtual Reality: A Comparative Evaluation of Joystick, Teleport, and Leaning , 2021, IEEE Transactions on Visualization and Computer Graphics.

[12]  Betsy Williams Sanders,et al.  Myo arm: swinging to explore a VE , 2015, SAP.

[13]  Huiyue Wu,et al.  User-Defined Body Gestures for TV-based Applications , 2012, 2012 Fourth International Conference on Digital Home.

[14]  Rudy Darken,et al.  The omni-directional treadmill: a locomotion device for virtual worlds , 1997, UIST '97.

[15]  Kai Kunze,et al.  Armswing: using arm swings for accessible and immersive navigation in AR/VR spaces , 2017, MUM.

[16]  Fabrizio Lamberti,et al.  On the Usability of Consumer Locomotion Techniques in Serious Games: Comparing Arm Swinging, Treadmills and Walk-in-Place , 2019, 2019 IEEE 9th International Conference on Consumer Electronics (ICCE-Berlin).

[17]  Tamás D. Gedeon,et al.  Touchless Gestural Interaction with Wizard-of-Oz: Analysing User Behaviour , 2015, OZCHI.

[18]  Max Mühlhäuser,et al.  Podoportation: Foot-Based Locomotion in Virtual Reality , 2020, CHI.

[19]  Naphtali Rishe,et al.  Gesture elicitation for 3D travel via multi-touch and mid-Air systems for procedurally generated pseudo-universe , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[20]  Stefania Serafin,et al.  Step aside: an initial exploration of gestural input for lateral movement during walking-in-place locomotion , 2018, VRST.

[21]  Keyur Sorathia,et al.  Elicitation Study of Body Gestures for Locomotion in HMD-VR Interfaces in a Sitting-Position , 2019, MIG.

[22]  Stefania Serafin,et al.  Tapping-In-Place: Increasing the naturalness of immersive walking-in-place locomotion through novel gestural input , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[23]  Bernhard E. Riecke,et al.  To move or not to move: can active control and user-driven motion cueing enhance self-motion perception ("vection") in virtual reality? , 2012, SAP.

[24]  Doug A. Bowman,et al.  A methodology for the evaluation of travel techniques for immersive virtual environments , 1998, Virtual Reality.

[25]  Betsy Williams Sanders,et al.  Human joystick: Wii-leaning to translate in large virtual environments , 2014, VRCAI '14.

[26]  Joaquim A. Jorge,et al.  Hip-directed walking-in-place using a single depth camera , 2017, Int. J. Hum. Comput. Stud..

[27]  Sayan Sarcar,et al.  Designing Mid-Air TV Gestures for Blind People Using User- and Choice-Based Elicitation Approaches , 2016, Conference on Designing Interactive Systems.

[28]  Tuncay Cakmak,et al.  Cyberith virtualizer: a locomotion device for virtual reality , 2014, SIGGRAPH '14.

[29]  Betsy Williams Sanders,et al.  Evaluation of walking in place on a Wii balance board to explore a virtual environment , 2011, TAP.

[30]  Anne Marie Piper,et al.  A Wizard-of-Oz elicitation study examining child-defined gestures with a whole-body interface , 2013, IDC.

[31]  Eelke Folmer,et al.  Legomotion: scalable walking-based virtual locomotion , 2017, VRST.

[32]  Donald A. Norman,et al.  Natural user interfaces are not natural , 2010, INTR.

[33]  Rahmita Wirza O. K. Rahmat,et al.  Swing-In-Place (SIP): A Less Fatigue Walking-in-Place Method With Side-Viewing Functionality for Mobile Virtual Reality , 2019, IEEE Access.

[34]  Mingyu Kim,et al.  A Study on Immersion and VR Sickness in Walking Interaction for Immersive Virtual Reality Applications , 2017, Symmetry.

[35]  Carman Neustaedter,et al.  NaviChair: Evaluating an Embodied Interface Using a Pointing Task to Navigate Virtual Reality , 2015, SUI.

[36]  Betsy Williams Sanders,et al.  VR locomotion: walking > walking in place > arm swinging , 2016, VRCAI.

[37]  Mary C. Whitton,et al.  GUD WIP: Gait-Understanding-Driven Walking-In-Place , 2010, 2010 IEEE Virtual Reality Conference (VR).

[38]  Xiangshi Ren,et al.  Designing concurrent full-body gestures for intense gameplay , 2015, Int. J. Hum. Comput. Stud..

[39]  A. B. Drought,et al.  WALKING PATTERNS OF NORMAL MEN. , 1964, The Journal of bone and joint surgery. American volume.

[40]  Sunghyuk Kwon,et al.  Can User-Derived Gesture be Considered as the Best Gesture for a Command?: Focusing on the Commands for Smart Home System , 2012 .

[41]  Adam S. Williams,et al.  Selection and Manipulation Whole-Body Gesture Elicitation Study In Virtual Reality , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[42]  Yu Wang,et al.  Influence of cultural factors on freehand gesture design , 2020, Int. J. Hum. Comput. Stud..

[43]  Sang Chul Ahn,et al.  A Walking-in-Place Method for Virtual Reality Using Position and Orientation Tracking , 2018, Sensors.

[44]  Mary C. Whitton,et al.  LLCM-WIP: Low-Latency, Continuous-Motion Walking-in-Place , 2008, 2008 IEEE Symposium on 3D User Interfaces.

[45]  Mary C. Whitton,et al.  Walking > walking-in-place > flying, in virtual environments , 1999, SIGGRAPH.

[46]  Mary C. Whitton,et al.  LUTE: A Locomotion Usability Test Environmentfor Virtual Reality , 2018, 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games).

[47]  Per Ola Kristensson,et al.  Memorability of pre-designed and user-defined gesture sets , 2013, CHI.

[48]  Ci Wang,et al.  User-Defined Gestures for Gestural Interaction: Extending from Hands to Other Body Parts , 2018, Int. J. Hum. Comput. Interact..

[49]  Bernhard E. Riecke,et al.  Comparing leaning-based motion cueing interfaces for virtual reality locomotion , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[50]  Motoki Kouzaki,et al.  Walk-, run- and gallop-like gait patterns in human sideways locomotion. , 2013, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[51]  Mel Slater,et al.  The Virtual Treadmill: A Naturalistic Metaphor for Navigation in Immersive Virtual Environments , 1995, Virtual Environments.

[52]  V. Ferrario,et al.  Active range of motion of the head and cervical spine: a three‐dimensional investigation in healthy young adults , 2002, Journal of orthopaedic research : official publication of the Orthopaedic Research Society.

[53]  T. Oberg,et al.  Basic gait parameters: reference data for normal subjects, 10-79 years of age. , 1993, Journal of rehabilitation research and development.

[54]  Valentina Gatteschi,et al.  An Evaluation Testbed for Locomotion in Virtual Reality , 2020, IEEE Transactions on Visualization and Computer Graphics.

[55]  Andy Cockburn,et al.  User-defined gestures for augmented reality , 2013, INTERACT.

[56]  Patricia S. Denbrook,et al.  Virtual Locomotion: Walking in Place through Virtual Environments , 1999, Presence.

[57]  Sang Chul Ahn,et al.  MIP-VR: An omnidirectional navigation and jumping method for VR shooting game using IMU , 2019, 2019 IEEE International Conference on Consumer Electronics (ICCE).

[58]  Alessandro De Luca,et al.  CyberWalk: Enabling unconstrained omnidirectional walking through virtual environments , 2008, ACM Trans. Appl. Percept..

[59]  Joaquim A. Jorge,et al.  A New Approach to Walking in Place , 2013, INTERACT.

[60]  G. A. Dean,et al.  AN ANALYSIS OF THE ENERGY EXPENDITURE IN LEVEL AND GRADE WALKING , 1965 .

[61]  Betsy Williams Sanders,et al.  Walking in place using the Microsoft Kinect to explore a large VE , 2013, SAP.

[62]  Ronald R. Mourant,et al.  Comparison of Simulator Sickness Using Static and Dynamic Walking Simulators , 2001 .

[63]  Kosuke Sato,et al.  LazyNav: 3D ground navigation with non-critical body parts , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[64]  Radu-Daniel Vatavu,et al.  Formalizing Agreement Analysis for Elicitation Studies: New Measures, Significance Test, and Toolkit , 2015, CHI.

[65]  Meredith Ringel Morris,et al.  User-defined gestures for surface computing , 2009, CHI.

[66]  M. P. Murray,et al.  Walking patterns of normal women. , 1970, Archives of physical medicine and rehabilitation.

[67]  Stefania Serafin,et al.  The influence of step frequency on the range of perceptually natural visual walking speeds during walking-in-place and treadmill locomotion , 2014, VRST '14.

[68]  Xiaolong Zhang,et al.  User-defined gesture interaction for immersive VR shopping applications , 2018, Behav. Inf. Technol..