Combining Dynamic Passive Haptics and Haptic Retargeting for Enhanced Haptic Feedback in Virtual Reality

To provide immersive haptic experiences, proxy-based haptic feedback systems for virtual reality (VR) face two central challenges: (1) similarity, and (2) colocation. While to solve challenge (1), physical proxy objects need to be sufficiently similar to their virtual counterparts in terms of haptic properties, for challenge (2), proxies and virtual counterparts need to be sufficiently colocated to allow for seamless interactions. To solve these challenges, past research introduced, among others, two successful techniques: (a) Dynamic Passive Haptic Feedback (DPHF), a hardware-based technique that leverages actuated props adapting their physical state during the VR experience, and (b) Haptic Retargeting, a software-based technique leveraging hand redirection to bridge spatial offsets between real and virtual objects. Both concepts have, up to now, not ever been studied in combination. This paper proposes to combine both techniques and reports on the results of a perceptual and a psychophysical experiment situated in a proof-of-concept scenario focused on the perception of virtual weight distribution. We show that users in VR overestimate weight shifts and that, when DPHF and HR are combined, significantly greater shifts can be rendered, compared to using only a weight-shifting prop or unnoticeable hand redirection. Moreover, we find the combination of DPHF and HR to let significantly larger spatial dislocations of proxy and virtual counterpart go unnoticed by users. Our investigation is the first to show the value of combining DPHF and HR in practice, validating that their combination can better solve the challenges of similarity and colocation than the individual techniques can do alone.

[1]  Robert S. Kennedy,et al.  Simulator Sickness Questionnaire: An enhanced method for quantifying simulator sickness. , 1993 .

[2]  Yifei Zhang,et al.  Dexmo: An Inexpensive and Lightweight Mechanical Exoskeleton for Motion Capture and Force Feedback in VR , 2016, CHI.

[3]  Frederick P. Brooks,et al.  Redirected Touching , 2013 .

[4]  Simon Richir,et al.  Influence of control/display ratio on the perception of mass of manipulated objects in virtual environments , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[5]  J. Gibson,et al.  Adaptation, after-effect and contrast in the perception of curved lines. , 1933 .

[6]  Antonio Krüger,et al.  Enhancing Texture Perception in Virtual Reality Using 3D-Printed Hair Structures , 2019, CHI.

[7]  Hunter G. Hoffman,et al.  Physically touching virtual objects using tactile augmentation enhances the realism of virtual environments , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[8]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[9]  Antonio Krüger,et al.  Shifty: A Weight-Shifting Dynamic Passive Haptic Proxy to Enhance Object Perception in Virtual Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.

[10]  E. Williams Experimental Designs Balanced for the Estimation of Residual Effects of Treatments , 1949 .

[11]  Matthias Harders,et al.  Adaptive Space Warping to Enhance Passive Haptics in an Arthroscopy Surgical Simulator , 2013, IEEE Transactions on Visualization and Computer Graphics.

[12]  Faisal Mushtaq,et al.  Studying human behavior with virtual reality: The Unity Experiment Framework , 2019, Behavior research methods.

[13]  Gerd Bruder,et al.  Estimation of Detection Thresholds for Redirected Walking Techniques , 2010, IEEE Transactions on Visualization and Computer Graphics.

[14]  Eric D. Ragan,et al.  Evaluating Remapped Physical Reach for Hand Interactions with Passive Haptics in Virtual Reality , 2018, IEEE Transactions on Visualization and Computer Graphics.

[15]  Mary C. Whitton,et al.  Passive haptics significantly enhances virtual environments , 2001 .

[16]  Eyal Ofek,et al.  Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences , 2016, CHI.

[17]  Maud Marchal,et al.  Toward "Pseudo-Haptic Avatars": Modifying the Visual Animation of Self-Avatar Can Simulate the Perception of Weight Lifting , 2014, IEEE Transactions on Visualization and Computer Graphics.

[18]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[19]  Takuji Narumi,et al.  Transcalibur: A Weight Shifting Virtual Reality Controller for 2D Shape Rendering based on Computational Perception Model , 2019, CHI.

[20]  Grigore C. Burdea,et al.  The Rutgers Master II-ND force feedback glove , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[21]  André Zenner,et al.  Shifting & Warping: A Case for the Combined Use of Dynamic Passive Haptics and Haptic Retargeting in VR , 2020, UIST.

[22]  K. Hinckley,et al.  Passive real-world interface props for neurosurgical visualization , 1994, CHI '94.

[23]  Hans-Werner Gellersen,et al.  Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences , 2015, CHI.

[24]  Marco Speicher,et al.  A Virtual Reality Couch Configurator Leveraging Passive Haptic Feedback , 2020, CHI Extended Abstracts.

[25]  Kazuki Takashima,et al.  Escape Room in Mixed Reality: 10th Annual 3DUI Contest , 2019, VR.

[26]  Sharif Razzaque,et al.  Redirected Walking , 2001, Eurographics.

[27]  André Zenner,et al.  Immersive Process Model Exploration in Virtual Reality , 2020, IEEE Transactions on Visualization and Computer Graphics.

[28]  Kyle Johnsen,et al.  Performance Benefits of High-Fidelity Passive Haptic Feedback in Virtual Reality Training , 2018, SUI.

[29]  Yuichi Ohta,et al.  Sensory Properties in Fusion of Visual/Haptic Stimuli Using Mixed Reality , 2010 .

[30]  Sean Follmer,et al.  Visuo-Haptic Illusions for Improving the Perceived Performance of Shape Displays , 2018, CHI.

[31]  Eric Burns,et al.  Combining passive haptics with redirected walking , 2005, ICAT '05.

[32]  Gerard Jounghyun Kim,et al.  Effects of sizes and shapes of props in tangible augmented reality , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[33]  Sean Follmer,et al.  Wolverine: A Wearable Haptic Interface for Grasping in VR , 2016, UIST.

[34]  Bing-Yu Chen,et al.  ElastOscillation: 3D Multilevel Force Feedback for Damped Oscillation on VR Controllers , 2020, CHI.

[35]  André Zenner,et al.  Envisioning Haptic Design for Immersive Virtual Environments , 2020, Conference on Designing Interactive Systems.

[36]  Mel Slater,et al.  Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[37]  Sriram Subramanian,et al.  Erg-O: Ergonomic Optimization of Immersive Virtual Environments , 2017, UIST.

[38]  Klaus H. Hinrichs,et al.  Taxonomy and Implementation of Redirection Techniques for Ubiquitous Passive Haptic Feedback , 2008, 2008 International Conference on Cyberworlds.

[39]  Enrico Rukzio,et al.  Breaking the Tracking: Enabling Weight Perception using Perceivable Tracking Offsets , 2018, CHI.

[40]  Dongjun Lee,et al.  Enlarging just noticeable differences of visual-proprioceptive conflict in VR using haptic feedback , 2015, 2015 IEEE World Haptics Conference (WHC).

[41]  Elia Gatti,et al.  Pseudo-Haptic Weight: Changing the Perceived Weight of Virtual Objects By Manipulating Control-Display Ratio , 2019, CHI.

[42]  Mel Slater,et al.  Depth of Presence in Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[43]  Andrey Krekhov,et al.  Self-Transforming Controllers for Virtual Reality First Person Shooters , 2017, CHI PLAY.

[44]  Antonio Krüger,et al.  Drag:on: A Virtual Reality Controller Providing Haptic Feedback Based on Drag and Weight Shift , 2019, CHI.

[45]  Andreas L. Lohse,et al.  Leveraging Change Blindness for Haptic Remapping in Virtual Environments , 2019, 2019 IEEE 5th Workshop on Everyday Virtual Reality (WEVR).

[46]  M. Turvey Dynamic touch. , 1996, The American psychologist.

[47]  Takuji Narumi,et al.  Computational design of hand-held VR controllers using haptic shape illusion , 2017, VRST.

[48]  Frederick A. A. Kingdom,et al.  Chapter 5 – Adaptive Methods∗ , 2016 .

[49]  Eyal Ofek,et al.  Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller , 2018, CHI.

[50]  Mary C. Whitton,et al.  15 Years of Research on Redirected Walking in Immersive Virtual Environments , 2018, IEEE Computer Graphics and Applications.

[51]  André Zenner Enhancing Proxy-Based Haptics in Virtual Reality , 2020, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW).

[52]  Martin Feick,et al.  The Virtual Reality Questionnaire Toolkit , 2020, UIST.

[53]  J. Landay,et al.  Beyond The Force: Using Quadcopters to Appropriate Objects and the Environment for Haptics in Virtual Reality , 2019, CHI.

[54]  Mounia Ziat,et al.  Enhancing virtual immersion through tactile feedback , 2014, UIST.

[55]  Sean Follmer,et al.  A Functional Optimization Based Approach for Continuous 3D Retargeted Touch of Arbitrary, Complex Boundaries in Haptic Virtual Reality , 2018, CHI.

[56]  Eyal Ofek,et al.  Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop , 2017, CHI.

[57]  Stefania Serafin,et al.  Knock on Wood: Combining Redirected Touching and Physical Props for Tool-Based Interaction in Virtual Reality , 2020, CHI.

[58]  Antonio Krüger,et al.  Estimating Detection Thresholds for Desktop-Scale Hand Redirection in Virtual Reality , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[59]  Run Yu,et al.  Pseudo-Haptic Display of Mass and Mass Distribution During Object Rotation in Virtual Reality , 2020, IEEE Transactions on Visualization and Computer Graphics.

[60]  Adalberto L. Simeone,et al.  Substitutional reality: Towards a research agenda , 2015, 2015 IEEE 1st Workshop on Everyday Virtual Reality (WEVR).

[61]  Antonio Krüger,et al.  Exploring Rock Climbing in Mixed Reality Environments , 2017, CHI Extended Abstracts.

[62]  Sean Follmer,et al.  Investigating the Detection of Bimanual Haptic Retargeting in Virtual Reality , 2019, VRST.

[63]  Josef B. Spjut,et al.  A variable shape and variable stiffness controller for haptic virtual interactions , 2018, 2018 IEEE International Conference on Soft Robotics (RoboSoft).

[64]  Thijs Roumen,et al.  TurkDeck: Physical Virtual Reality Based on People , 2015, UIST.