Effect of Cutaneous Feedback on the Perception of Virtual Object Weight during Manipulation

Haptic interface technologies for virtual reality applications have been developed to increase the reality and manipulability of a virtual object by creating a diverse tactile sensation. Most evaluation of the haptic technologies, however, have been limited to the haptic perception of the tactile stimuli via static virtual objects. Noting this, we investigated the effect of lateral cutaneous feedback, along with kinesthetic feedback on the perception of virtual object weight during manipulation. We modeled the physical interaction between a participant’s finger avatars and virtual objects. The haptic stimuli were rendered with custom-built haptic feedback systems that can provide kinesthetic and lateral cutaneous feedback to the participant. We conducted two virtual object manipulation experiments, 1. a virtual object manipulation with one finger, and 2. the pull-out and lift-up of a virtual object grasped with a precision grip. The results of Experiment 1 indicate that the participants felt the virtual object rendered with lateral cutaneous feedback significantly heavier than with only kinesthetic feedback ( p  < 0.05 for m ref  = 100 and 200 g). Similarly, the participants of Experiment 2 felt the virtual objects significantly heavier when lateral cutaneous feedback was available ( p  < 0.05 for m ref  = 100, 200, and 300 g). Therefore, the additional lateral cutaneous feedback to the force feedback led the participants to feel the virtual object heavier than without the cutaneous feedback. The results also indicate that the contact force applied to a virtual object during manipulation can be a function of the perceived object weight ( p  = 0.005 for Experiment 1 and p  = 0.02 for Experiment 2).

[1]  James M. Hillis,et al.  Slant from texture and disparity cues: optimal cue combination. , 2004, Journal of vision.

[2]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[3]  M. Srinivasan,et al.  Tactile discrimination of shape: responses of slowly and rapidly adapting mechanoreceptive afferents to a step indented into the monkey fingerpad , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[4]  Philippe Lefèvre,et al.  Perception of partial slips under tangential loading of the fingertip , 2018, Scientific Reports.

[5]  Lee A Baugh,et al.  Material evidence: interaction of well-learned priors and sensorimotor memory when lifting objects. , 2012, Journal of neurophysiology.

[6]  Hong Z. Tan,et al.  Effect of Cutaneous Feedback on the Perceived Hardness of a Virtual Object , 2018, IEEE Transactions on Haptics.

[7]  Carlos Canudas de Wit,et al.  Friction Models and Friction Compensation , 1998, Eur. J. Control.

[8]  John Kenneth Salisbury,et al.  A constraint-based god-object method for haptic display , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[9]  P. Berkelman,et al.  Design of a Hemispherical Magnetic Levitation Haptic Interface Device , 1996, Dynamic Systems and Control.

[10]  Joonbum Bae,et al.  A Force-Controllable Compact Actuator Module for a Wearable Hand Exoskeleton , 2014 .

[11]  Antonio Frisoli,et al.  A 3-RSR Haptic Wearable Device for Rendering Fingertip Contact Forces , 2017, IEEE Transactions on Haptics.

[12]  Physical Modeling of Mechanical Friction in Simulink , 2008 .

[13]  George G. Adams,et al.  Contact mechanics and friction , 2005 .

[14]  Pinhas Ben-Tzvi,et al.  Sensing and Force-Feedback Exoskeleton (SAFE) Robotic Glove , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  R. Johansson,et al.  Control of Grip Force When Tilting Objects: Effect of Curvature of Grasped Surfaces and Applied Tangential Torque , 1998, The Journal of Neuroscience.

[16]  H. Levitt Transformed up-down methods in psychoacoustics. , 1971, The Journal of the Acoustical Society of America.

[17]  Yonghwan Oh,et al.  High-Force Display Capability and Wide Workspace With a Novel Haptic Interface , 2017, IEEE/ASME Transactions on Mechatronics.

[18]  Allison M. Okamura,et al.  Augmentation Of Stiffness Perception With a 1-Degree-of-Freedom Skin Stretch Device , 2014, IEEE Transactions on Human-Machine Systems.

[19]  Antonio Frisoli,et al.  The contribution of cutaneous and kinesthetic sensory modalities in haptic perception of orientation , 2011, Brain Research Bulletin.

[20]  Matteo Bianchi,et al.  Rendering Softness: Integration of Kinesthetic and Cutaneous Information in a Haptic Device , 2010, IEEE Transactions on Haptics.

[21]  Alex Olwal,et al.  Grabity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality , 2017, UIST.

[22]  Mandayam A. Srinivasan,et al.  Encoding of Shape in the Responses of Cutaneous Mechanoreceptors , 1991 .

[23]  Sean Follmer,et al.  Wolverine: A wearable haptic interface for grasping in virtual reality , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[24]  Mark T. Wallace,et al.  Increased Neural Strength and Reliability to Audiovisual Stimuli at the Boundary of Peripersonal Space , 2019, Journal of Cognitive Neuroscience.

[25]  Robert A Jacobs,et al.  Bayesian integration of visual and auditory signals for spatial localization. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[26]  A. Wing,et al.  Anticipating load torques produced by voluntary movements. , 1998, Journal of experimental psychology. Human perception and performance.

[27]  R. Johansson,et al.  Tangential torque effects on the control of grip forces when holding objects with a precision grip. , 1997, Journal of neurophysiology.

[28]  S. Gandevia,et al.  Sensory integration in the perception of movements at the human metacarpophalangeal joint , 2000, The Journal of physiology.

[29]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[30]  R. Johansson,et al.  Experience Can Change Distinct Size-Weight Priors Engaged in Lifting Objects and Judging their Weights , 2008, Current Biology.

[31]  Oussama Khatib,et al.  Haptic display for human interaction with virtual dynamic environments , 2001, J. Field Robotics.

[32]  Claudio Pacchierotti,et al.  A Three Revolute-Revolute-Spherical Wearable Fingertip Cutaneous Device for Stiffness Rendering , 2018, IEEE Transactions on Haptics.

[33]  Jaeyoung Park,et al.  Effect of Haptic Feedback on the Perceived Size of a Virtual Object , 2019, IEEE Access.

[34]  Jee-Hwan Ryu,et al.  Portable Exoskeleton Glove With Soft Structure for Hand Assistance in Activities of Daily Living , 2017, IEEE/ASME Transactions on Mechatronics.

[35]  A. Wing,et al.  Anticipating load torques produced by voluntary movements. , 1998, Journal of Experimental Psychology: Human Perception and Performance.

[36]  Allison M. Okamura,et al.  Sensory Substitution and Augmentation Using 3-Degree-of-Freedom Skin Deformation Feedback , 2015, IEEE Transactions on Haptics.

[37]  J. Edward Colgate,et al.  Two experiments on the perception of slip at the fingertip , 2004, 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2004. HAPTICS '04. Proceedings..

[38]  Matteo Bianchi,et al.  Tactile slip and hand displacement: Bending hand motion with tactile illusions , 2017, 2017 IEEE World Haptics Conference (WHC).

[39]  Sungchul Kang,et al.  Micro hydraulic system using slim artificial muscles for a wearable haptic glove , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  David S. Ebert,et al.  Haptic Identification of Stiffness and Force Magnitude , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[41]  Kenneth O. Johnson,et al.  The roles and functions of cutaneous mechanoreceptors , 2001, Current Opinion in Neurobiology.

[42]  Tsukasa Ogasawara,et al.  Weight and friction display device by controlling the slip condition of a fingertip , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[43]  Keijiro Yamamoto,et al.  Toward interdisciplinary cooperation in visualization , 2001 .

[44]  Thomas H. Massie,et al.  The PHANToM Haptic Interface: A Device for Probing Virtual Objects , 1994 .

[45]  Maud Marchal,et al.  HapTip: Displaying Haptic Shear Forces at the Fingertips for Multi-Finger Interaction in Virtual Environments , 2016, Front. ICT.

[46]  Ian R Summers,et al.  A broadband tactile array on the fingertip. , 2002, The Journal of the Acoustical Society of America.

[47]  M. Guerraz,et al.  Optimal visuotactile integration for velocity discrimination of self-hand movements. , 2016, Journal of neurophysiology.

[48]  M. Srinivasan,et al.  Tactile detection of slip: surface microgeometry and peripheral neural codes. , 1990, Journal of neurophysiology.

[49]  R. J. van Beers,et al.  Integration of proprioceptive and visual position-information: An experimentally supported model. , 1999, Journal of neurophysiology.

[50]  Marek Wojtyra,et al.  Modeling of static friction in closed-loop kinematic chains—Uniqueness and parametric sensitivity problems , 2017 .

[51]  Allison M. Okamura,et al.  Three-Dimensional Skin Deformation as Force Substitution: Wearable Device Design and Performance During Haptic Exploration of Virtual Environments , 2017, IEEE Transactions on Haptics.

[52]  Mark R. Cutkosky,et al.  Contact Location Display for Haptic Perception of Curvature and Object Motion , 2005, Int. J. Robotics Res..

[53]  H. Forssberg,et al.  Lighter or Heavier Than Predicted: Neural Correlates of Corrective Mechanisms during Erroneously Programmed Lifts , 2006, The Journal of Neuroscience.