Object discrimination using electrotactile feedback

OBJECTIVE A variety of bioengineering systems are being developed to restore tactile sensations in individuals who have lost somatosensory feedback because of spinal cord injury, stroke, or amputation. These systems typically detect tactile force with sensors placed on an insensate hand (or prosthetic hand in the case of amputees) and deliver touch information by electrically or mechanically stimulating sensate skin above the site of injury. Successful object manipulation, however, also requires proprioceptive feedback representing the configuration and movements of the hand and digits. APPROACH Therefore, we developed a simple system that simultaneously provides information about tactile grip force and hand aperture using current amplitude-modulated electrotactile feedback. We evaluated the utility of this system by testing the ability of eight healthy human subjects to distinguish among 27 objects of varying sizes, weights, and compliances based entirely on electrotactile feedback. The feedback was modulated by grip-force and hand-aperture sensors placed on the hand of an experimenter (not visible to the subject) grasping and lifting the test objects. We were also interested to determine the degree to which subjects could learn to use such feedback when tested over five consecutive sessions. MAIN RESULTS The average percentage correct identifications on day 1 (28.5%  ±  8.2% correct) was well above chance (3.7%) and increased significantly with training to 49.2%  ±  10.6% on day 5. Furthermore, this training transferred reasonably well to a set of novel objects. SIGNIFICANCE These results suggest that simple, non-invasive methods can provide useful multisensory feedback that might prove beneficial in improving the control over prosthetic limbs.

[1]  Soohyun Kim,et al.  Mixed-Modality Stimulation to Evoke Two Modalities Simultaneously in One Channel for Electrocutaneous Sensory Feedback , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  C. Sherrington,et al.  VIII. Experiments upon the influence of sensory nerves upon movement and nutrition of the limbs. Preliminary communication , 1985, Proceedings of the Royal Society of London.

[3]  Benjamin C. K. Tee,et al.  25th Anniversary Article: The Evolution of Electronic Skin (E‐Skin): A Brief History, Design Considerations, and Recent Progress , 2013, Advanced materials.

[4]  Shuzhi Sam Ge,et al.  Prosthetic finger phalanges with lifelike skin compliance for low-force social touching interactions , 2011, Journal of NeuroEngineering and Rehabilitation.

[5]  Andrew Y. J. Szeto,et al.  Electrocutaneous code pairs for artificial sensory communication systems , 2006, Annals of Biomedical Engineering.

[6]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[7]  Peter H. Veltink,et al.  Stiffness Feedback for Myoelectric Forearm Prostheses Using Vibrotactile Stimulation , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  Thierry Keller,et al.  Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis , 2016 .

[9]  D. Szarowski,et al.  Cerebral Astrocyte Response to Micromachined Silicon Implants , 1999, Experimental Neurology.

[10]  A. M. Lassek.  INACTIVATION OF VOLUNTARY MOTOR FUNCTION FOLLOWING RHIZOTOMY , 1953, Journal of neuropathology and experimental neurology.

[11]  Hannes P. Saal,et al.  Biomimetic approaches to bionic touch through a peripheral nerve interface , 2015, Neuropsychologia.

[12]  Dario Farina,et al.  Virtual Grasping: Closed-Loop Force Control Using Electrotactile Feedback , 2014, Comput. Math. Methods Medicine.

[13]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[14]  Nicholas V. Annetta,et al.  Restoring cortical control of functional movement in a human with quadriplegia , 2016, Nature.

[15]  T. Twitchell,et al.  Sensory factors in purposive movement. , 1954, Journal of neurophysiology.

[16]  K. Horch,et al.  Object Discrimination With an Artificial Hand Using Electrical Stimulation of Peripheral Tactile and Proprioceptive Pathways With Intrafascicular Electrodes , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  R. S. Johansson,et al.  Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects , 2004, Experimental Brain Research.

[18]  N. Thakor,et al.  Journal of Neuroengineering and Rehabilitation Open Access a Brain-computer Interface with Vibrotactile Biofeedback for Haptic Information , 2007 .

[19]  Christian Antfolk,et al.  Sensory feedback in upper limb prosthetics , 2013, Expert review of medical devices.

[20]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[21]  R. Andersen,et al.  Decoding motor imagery from the posterior parietal cortex of a tetraplegic human , 2015, Science.

[22]  W.J. Tompkins,et al.  Electrotactile and vibrotactile displays for sensory substitution systems , 1991, IEEE Transactions on Biomedical Engineering.

[23]  Joseph E O'Doherty,et al.  A learning–based approach to artificial sensory feedback leads to optimal integration , 2014, Nature Neuroscience.

[24]  M. L. Nicolelis,et al.  Assimilation of virtual legs and perception of floor texture by complete paraplegic patients receiving artificial tactile feedback , 2016, Scientific Reports.

[25]  T. T. Norton,et al.  Optic tract lesions sparing pattern vision in cats. , 1967, Experimental neurology.

[26]  L. Miller,et al.  Restoring sensorimotor function through intracortical interfaces: progress and looming challenges , 2014, Nature Reviews Neuroscience.

[27]  John Lyman,et al.  Comparison of codes for sensory feedback using electrocutaneous tracking , 1977, Annals of Biomedical Engineering.

[28]  Thierry Keller,et al.  Multichannel Electrotactile Feedback With Spatial and Mixed Coding for Closed-Loop Control of Grasping Force in Hand Prostheses , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Thierry Keller,et al.  Short- and Long-Term Learning of Feedforward Control of a Myoelectric Prosthesis with Sensory Feedback by Amputees , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[30]  Francis R. Willett,et al.  Restoration of reaching and grasping in a person with tetraplegia through brain-controlled muscle stimulation: a proof-of-concept demonstration , 2017, The Lancet.

[31]  R. J. Vogelstein,et al.  Restoring the sense of touch with a prosthetic hand through a brain interface , 2013, Proceedings of the National Academy of Sciences.

[32]  A. Wing,et al.  Active touch sensing , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[33]  C E Chapman,et al.  Active versus passive touch: factors influencing the transmission of somatosensory signals to primary somatosensory cortex. , 1994, Canadian journal of physiology and pharmacology.

[34]  E. Taub,et al.  Deafferentation in monkeys: Pointing at a target without visual feedback , 1975, Experimental Neurology.

[35]  A. M. Lassek.  Potency of Isolated Brachial Dorsal Roots in Controlling Muscular Physiology , 1953, Neurology.

[36]  Stephen T. Foldes,et al.  Intracortical microstimulation of human somatosensory cortex , 2016, Science Translational Medicine.

[37]  Nicolas Y. Masse,et al.  Reach and grasp by people with tetraplegia using a neurally controlled robotic arm , 2012, Nature.

[38]  Blair A. Lock,et al.  Redirection of cutaneous sensation from the hand to the chest skin of human amputees with targeted reinnervation , 2007, Proceedings of the National Academy of Sciences.

[39]  J. Webster,et al.  Maximal dynamic range electrotactile stimulation waveforms , 1992, IEEE Transactions on Biomedical Engineering.

[40]  C. Cipriani,et al.  HyVE: Hybrid Vibro-Electrotactile Stimulation for Sensory Feedback and Substitution in Rehabilitation , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[41]  R R Riso,et al.  Strategies for providing upper extremity amputees with tactile and hand position feedback--moving closer to the bionic arm. , 1999, Technology and health care : official journal of the European Society for Engineering and Medicine.

[42]  Spencer Kellis,et al.  A cognitive neuroprosthetic that uses cortical stimulation for somatosensory feedback , 2014, Journal of neural engineering.

[43]  M. Cynader,et al.  Somatosensory cortical map changes following digit amputation in adult monkeys , 1984, The Journal of comparative neurology.

[44]  José Pascual Molina,et al.  Identifying Virtual 3D Geometric Shapes with a Vibrotactile Glove , 2016, IEEE Computer Graphics and Applications.

[45]  Dario Farina,et al.  Multichannel electrotactile feedback for simultaneous and proportional myoelectric control , 2016, Journal of neural engineering.

[46]  David C. Martin,et al.  Neuronal cell loss accompanies the brain tissue response to chronically implanted silicon microelectrode arrays , 2005, Experimental Neurology.

[47]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[48]  Daniel Tan,et al.  Sensory feedback by peripheral nerve stimulation improves task performance in individuals with upper limb loss using a myoelectric prosthesis , 2016, Journal of neural engineering.

[49]  S. Bensmaia,et al.  Behavioral assessment of sensitivity to intracortical microstimulation of primate somatosensory cortex , 2015, Proceedings of the National Academy of Sciences.

[50]  Frank A. Saunders,et al.  Electrocutaneous Stimulation for Sensory Communication in Rehabilitation Engineering , 1982, IEEE Transactions on Biomedical Engineering.

[51]  Hannes Bleuler,et al.  Active tactile exploration enabled by a brain-machine-brain interface , 2011, Nature.

[52]  A. Fuglevand,et al.  Perception of electrical and mechanical stimulation of the skin: implications for electrotactile feedback , 2009, Journal of neural engineering.

[53]  B L Richardson,et al.  Can passive touch be better than active touch? A comparison of active and passive tactile maze learning. , 1981, British journal of psychology.

[54]  P. Rossini,et al.  Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans , 2016, eLife.

[55]  Panagiotis K. Artemiadis,et al.  Object discrimination using optimized multi-frequency auditory cross-modal haptic feedback , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.