Sensory Substitution and Augmentation Using 3-Degree-of-Freedom Skin Deformation Feedback

During tool-mediated interaction with everyday objects, we experience kinesthetic forces and tactile sensations in the form of vibration and skin deformation at the fingerpad. Fingerpad skin deformation is caused by forces applied tangentially and normally to the fingerpad skin, resulting in tangential and normal skin displacement. We designed a device to convey 3-degree-of-freedom (DoF) force information to the user via skin deformation, and conducted two experiments to determine the devices effectiveness for force-feedback substitution and augmentation. For sensory substitution, participants used 1-DoF and 3-DoF skin deformation feedback to locate a feature in a 3-DoF virtual environment. Participants showed improved precision and shorter completion time when using 3-DoF compared to 1-DoF skin deformation feedback. For sensory augmentation, participants traced a path in space from an initial to a target location, while under guidance from force and/or skin deformation feedback. When force feedback was augmented with skin deformation, participants reduced their path-following error over the cases when force or skin deformation feedback are used separately. We conclude that 3-DoF skin deformation feedback is effective in substituting or augmenting force feedback. Such substitution or augmentation could be used when force feedback is unattainable or attenuated due to device limitations or system instability.

[1]  Mandayam A. Srinivasan,et al.  Tangential versus normal displacements of skin: relative effectiveness for producing tactile sensations , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[2]  S S Hsiao,et al.  Vibratory adaptation of cutaneous mechanoreceptive afferents. , 2005, Journal of neurophysiology.

[3]  Robert D. Howe,et al.  Tactile Display of Vibratory Information in Teleoperation and Virtual Environments , 1995, Presence: Teleoperators & Virtual Environments.

[4]  Allison M. Okamura,et al.  Sensory substitution using 3-degree-of-freedom tangential and normal skin deformation feedback , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[5]  William R. Provancher,et al.  Fingerpad Skin Stretch Increases the Perception of Virtual Friction , 2009, IEEE Transactions on Haptics.

[6]  Dale A. Lawrence.  Stability and transparency in bilateral teleoperation , 1993, IEEE Trans. Robotics Autom..

[7]  François Conti,et al.  CHAI: An Open-Source Library for the Rapid Development of Haptic Scenes , 2005 .

[8]  Claudio Pacchierotti,et al.  On the role of cutaneous force in teleoperation: subtracting kinesthesia from complete haptic feedback , 2013, 2013 World Haptics Conference (WHC).

[9]  M. López,et al.  Delta robot: Inverse, direct, and intermediate Jacobians , 2006 .

[10]  Caroline G. L. Cao,et al.  Vibrotactile force feedback system for minimally invasive surgical procedures , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[11]  Allison M. Okamura,et al.  Sensory substitution via cutaneous skin stretch feedback , 2013, 2013 IEEE International Conference on Robotics and Automation.

[12]  Thomas Hulin,et al.  Evaluation of a vibrotactile feedback device for spatial guidance , 2011, 2011 IEEE World Haptics Conference.

[13]  William R. Provancher,et al.  Haptic matching of directional force and skin stretch feedback cues , 2013, 2013 World Haptics Conference (WHC).

[14]  Roberta L. Klatzky,et al.  Good vibrations: Asymmetric vibrations for directional haptic cues , 2009, World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[15]  Yoji Yamada,et al.  Relative Contribution Ratios of Skin and Proprioceptive Sensations in Perception of Force Applied to Fingertip , 2014, IEEE Transactions on Haptics.

[16]  Allison M. Okamura,et al.  Reality-based models for vibration feedback in virtual environments , 2001 .

[17]  Pierre E. Dupont,et al.  Multichannel vibrotactile display for sensory substitution during teleoperation , 2002, SPIE Optics East.

[18]  Katherine J. Kuchenbecker,et al.  Characterizing the Human Wrist for Improved Haptic Interaction , 2003 .

[19]  Allison M. Okamura,et al.  Stable Forbidden-Region Virtual Fixtures for Bilateral Telemanipulation , 2006 .

[20]  Satoshi Tadokoro,et al.  Vibrotactile Stimuli Applied to Finger Pads as Biases for Perceived Inertial and Viscous Loads , 2011, IEEE Transactions on Haptics.

[21]  William R. Provancher,et al.  Back-to-back skin stretch feedback for communicating five degree-of-freedom direction cues , 2013, 2013 World Haptics Conference (WHC).

[22]  Claudio Pacchierotti,et al.  Sensory Subtraction in Robot-Assisted Surgery: Fingertip Skin Deformation Feedback to Ensure Safety and Improve Transparency in Bimanual Haptic Interaction , 2014, IEEE Transactions on Biomedical Engineering.

[23]  Katherine J. Kuchenbecker,et al.  Improving contact realism through event-based haptic feedback , 2006, IEEE Transactions on Visualization and Computer Graphics.

[24]  M. Goldfarb,et al.  The effect of virtual surface stiffness on the haptic perception of detail , 2004, IEEE/ASME Transactions on Mechatronics.

[25]  Marcia Kilchenman O'Malley,et al.  Tactile feedback of object slip improves performance in a grasp and hold task , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[26]  Mark R. Cutkosky,et al.  Rotational Skin Stretch Feedback: A Wearable Haptic Display for Motion , 2010, IEEE Transactions on Haptics.

[27]  Robert D. Howe,et al.  Multi-channel vibrotactile display for teleoperated assembly , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[28]  Rieko Osu,et al.  The central nervous system stabilizes unstable dynamics by learning optimal impedance , 2001, Nature.

[29]  Domenico Prattichizzo,et al.  Simplified design of haptic display by extending one-point kinesthetic feedback to multipoint tactile feedback , 2010, 2010 IEEE Haptics Symposium.

[30]  John Kenneth Salisbury,et al.  A constraint-based god-object method for haptic display , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[31]  Christopher R. Wagner,et al.  The Benefit of Force Feedback in Surgery: Examination of Blunt Dissection , 2007, PRESENCE: Teleoperators and Virtual Environments.

[32]  William R. Provancher,et al.  A smart phone peripheral with bi-manual skin stretch haptic feedback and user input , 2013, 2013 IEEE International Conference on Consumer Electronics (ICCE).

[33]  Charles A. Stewart,et al.  Improved Tactile Shear Feedback: Tactor Design and an Aperture-Based Restraint , 2011, IEEE Transactions on Haptics.

[34]  Allison M. Okamura,et al.  Augmentation Of Stiffness Perception With a 1-Degree-of-Freedom Skin Stretch Device , 2014, IEEE Transactions on Human-Machine Systems.

[35]  Claudio Pacchierotti,et al.  Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics , 2011, IEEE Transactions on Haptics.

[36]  Claudio Pacchierotti,et al.  Improving Transparency in Teleoperation by Means of Cutaneous Tactile Force Feedback , 2014, TAP.

[37]  Christopher R. Wagner,et al.  Mechanisms of performance enhancement with force feedback , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[38]  William R. Provancher,et al.  Perception of Direction for Applied Tangential Skin Displacement: Effects of Speed, Displacement, and Repetition , 2010, IEEE Transactions on Haptics.