Augmenting Perceived Softness of Haptic Proxy Objects Through Transient Vibration and Visuo-Haptic Illusion in Virtual Reality

In this work, we investigate the effects of active transient vibration and visuo-haptic illusion to augment the perceived softness of haptic proxy objects. We introduce a system combining active transient vibration at the fingertip with visuo-haptic illusions. In our hand-held device, a voice coil actuator transmits active transient vibrations to the index fingertip, while a force sensor measures the force applied on passive proxy objects to create visuo-haptic illusions in virtual reality. We conducted three user studies to understand both the vibrotactile effect and its combined effect with visuo-haptic illusions. A preliminary study confirmed that active transient vibrations can intuitively alter the perceived softness of a proxy object. Our first study demonstrated that those same active transient vibrations can generate different perceptions of softness depending on the material of the proxy object used. In our second study, we evaluated the combination of active transient vibration and visuo-haptic illusion, and found that both significantly influence perceived softness, with with the visuo-haptic effect being dominant. Our third study further investigated the vibrotactile effect while controlling for the visuo-haptic illusion. The combination of these two methods allows users to effectively perceive various levels of softness when interacting with haptic proxy objects.

[1]  Hong Z. Tan,et al.  HUMAN FACTORS FOR THE DESIGN OF FORCE-REFLECTING HAPTIC INTERFACES , 1994 .

[2]  Allison M. Okamura,et al.  Haptic jamming: A deformable geometry, variable stiffness tactile display using pneumatics and particle jamming , 2013, 2013 World Haptics Conference (WHC).

[3]  Hiroyuki Kajimoto,et al.  Augmentation of Material Property by Modulating Vibration Resulting from Tapping , 2012, EuroHaptics.

[4]  Jamie Kyujin Paik,et al.  Variable stiffness fabrics with embedded shape memory materials for wearable applications , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Eyal Ofek,et al.  NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers , 2016, UIST.

[6]  Sean Follmer,et al.  A Functional Optimization Based Approach for Continuous 3D Retargeted Touch of Arbitrary, Complex Boundaries in Haptic Virtual Reality , 2018, CHI.

[7]  Bulmaro A. Valdés,et al.  Determining the Accuracy of Oculus Touch Controllers for Motor Rehabilitation Applications Using Quantifiable Upper Limb Kinematics: Validation Study , 2019, JMIR Biomedical Engineering.

[8]  J. Gibson,et al.  Adaptation, after-effect and contrast in the perception of curved lines. , 1933 .

[9]  Lawrence H. Kim,et al.  Robotic Assembly of Haptic Proxy Objects for Tangible Interaction and Virtual Reality , 2017, ISS.

[10]  Allison M. Okamura,et al.  Sensory augmentation of stiffness using fingerpad skin stretch , 2013, 2013 World Haptics Conference (WHC).

[11]  Sriram Subramanian,et al.  Changibles: analyzing and designing shape changing constructive assembly , 2014, CHI.

[12]  Massimiliano Di Luca Multisensory Softness , 2014, Springer Series on Touch and Haptic Systems.

[13]  Antonio Bicchi,et al.  Haptic discrimination of softness in teleoperation: the role of the contact area spread rate , 2000, IEEE Trans. Robotics Autom..

[14]  M. A. Srinivassan The impact of visual information on the haptic perception of stiffness in virtual environments , 1996 .

[15]  Mar Gonzalez-Franco,et al.  The uncanny valley of haptics , 2018, Science Robotics.

[16]  Luv Kohli,et al.  Redirected touching: Warping space to remap passive haptics , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[17]  Daniel J. Wigdor,et al.  Annexing Reality: Enabling Opportunistic Use of Everyday Objects as Tangible Proxies in Augmented Reality , 2016, CHI.

[18]  Maud Marchal,et al.  The King-Kong Effects: Improving sensation of walking in VR with visual and tactile vibrations at each step , 2012, 2012 IEEE Symposium on 3D User Interfaces (3DUI).

[19]  Seokhee Jeon,et al.  Haptic Augmented Reality: Taxonomy and an Example of Stiffness Modulation , 2009, PRESENCE: Teleoperators and Virtual Environments.

[20]  Allison M. Okamura,et al.  Three-Dimensional Skin Deformation as Force Substitution: Wearable Device Design and Performance During Haptic Exploration of Virtual Environments , 2017, IEEE Transactions on Haptics.

[21]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[22]  Bruno L. Giordano Sound source perception in impact sounds , 2006 .

[23]  A. Diederich,et al.  Bimodal and trimodal multisensory enhancement: Effects of stimulus onset and intensity on reaction time , 2004, Perception & psychophysics.

[24]  Sean Follmer,et al.  shapeShift: A Mobile Tabletop Shape Display for Tangible and Haptic Interaction , 2017, UIST.

[25]  Abderrahmane Kheddar,et al.  Pseudo-haptic feedback: can isometric input devices simulate force feedback? , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[26]  Yoji Yamada,et al.  Vibration-based rendering of virtual hardness: Frequency characteristics of perception , 2017, 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE).

[27]  Maud Marchal,et al.  Virtual chromatic percussions simulated by pseudo-haptic and vibrotactile feedback , 2011, Advances in Computer Entertainment Technology.

[28]  Piet Lammertse,et al.  HapticMaster - a generic force controlled robot for human interaction , 2003, Ind. Robot.

[29]  R. Klatzky,et al.  Hand movements: A window into haptic object recognition , 1987, Cognitive Psychology.

[30]  Hiroyuki Kajimoto,et al.  Vibration Feedback Latency Affects Material Perception During Rod Tapping Interactions , 2017, IEEE Transactions on Haptics.

[31]  F.J. Canadas-Quesada,et al.  Improvement of perceived stiffness using auditory stimuli in haptic virtual reality , 2006, MELECON 2006 - 2006 IEEE Mediterranean Electrotechnical Conference.

[32]  Sean Follmer,et al.  Wolverine: A wearable haptic interface for grasping in virtual reality , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[33]  M. L. Demattè,et al.  Cross-modal interactions between olfaction and touch. , 2006, Chemical senses.

[34]  Maud Marchal,et al.  Elastic-Arm: Human-scale passive haptic feedback for augmenting interaction and perception in virtual environments , 2015, 2015 IEEE Virtual Reality (VR).

[35]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[36]  Lukas Kaim,et al.  Visual and haptic integration in the estimation of softness of deformable objects , 2013, i-Perception.

[37]  Shoichi Hasegawa,et al.  Characteristics of Perception of Stiffness by Varied Tapping Velocity and Penetration in Using Event-Based Haptic , 2009, EGVE/ICAT/EuroVR.

[38]  R H LaMotte,et al.  Softness discrimination with a tool. , 2000, Journal of neurophysiology.

[39]  Yoji Yamada,et al.  Psychophysical Dimensions of Tactile Perception of Textures , 2013, IEEE Transactions on Haptics.

[40]  M. Di Luca,et al.  Effects of visual–haptic asynchronies and loading–unloading movements on compliance perception , 2011, Brain Research Bulletin.

[41]  Lynette A. Jones,et al.  Application of Psychophysical Techniques to Haptic Research , 2013, IEEE Transactions on Haptics.

[42]  Jeremy R. Cooperstock,et al.  Vibration Influences Haptic Perception of Surface Compliance During Walking , 2011, PloS one.

[43]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[44]  Ravin Balakrishnan,et al.  An interface for virtual 3D sculpting via physical proxy , 2006, GRAPHITE '06.

[45]  Katherine J. Kuchenbecker,et al.  Improving contact realism through event-based haptic feedback , 2006, IEEE Transactions on Visualization and Computer Graphics.

[46]  Allison M. Okamura,et al.  Reality-based models for vibration feedback in virtual environments , 2001 .

[47]  Martin Buss,et al.  Combination and Integration in the Perception of Visual-Haptic Compliance Information , 2010, IEEE Transactions on Haptics.

[48]  M. Srinivasan,et al.  Tactual discrimination of softness. , 1995, Journal of neurophysiology.

[49]  Antonio Bicchi,et al.  Towards a Haptic Black Box for free-hand softness and shape exploration , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[50]  Eyal Ofek,et al.  Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop , 2017, CHI.

[51]  Federico Avanzini,et al.  Haptic-Auditory Rendering and Perception of Contact Stiffness , 2006, HAID.

[52]  A. Landi Human Hand Function , 2007 .

[53]  Xavier de Tinguy,et al.  Enhancing the Stiffness Perception of Tangible Objects in Mixed Reality Using Wearable Haptics , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[54]  Eyal Ofek,et al.  Haptic Retargeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences , 2016, CHI.

[55]  Allison M. Okamura,et al.  Vibration feedback models for virtual environments , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[56]  Matthias Harders,et al.  Influence of visual and haptic delays on stiffness perception in augmented reality , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[57]  Maud Marchal,et al.  FlexiFingers: Multi-finger interaction in VR combining passive haptics and pseudo-haptics , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[58]  Danilo De Rossi,et al.  Electroactive Elastomeric Haptic Displays of Organ Motility and Tissue Compliance for Medical Training and Surgical Force Feedback , 2009, IEEE Transactions on Biomedical Engineering.

[59]  Bing Wu,et al.  Force, Torque, and Stiffness: Interactions in Perceptual Discrimination , 2011, IEEE Transactions on Haptics.

[60]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.