Improving internal model strength and performance of prosthetic hands using augmented feedback

BackgroundThe loss of an arm presents a substantial challenge for upper limb amputees when performing activities of daily living. Myoelectric prosthetic devices partially replace lost hand functions; however, lack of sensory feedback and strong understanding of the myoelectric control system prevent prosthesis users from interacting with their environment effectively. Although most research in augmented sensory feedback has focused on real-time regulation, sensory feedback is also essential for enabling the development and correction of internal models, which in turn are used for planning movements and reacting to control variability faster than otherwise possible in the presence of sensory delays.MethodsOur recent work has demonstrated that audio-augmented feedback can improve both performance and internal model strength for an abstract target acquisition task. Here we use this concept in controlling a robotic hand, which has inherent dynamics and variability, and apply it to a more functional grasp-and-lift task. We assessed internal model strength using psychophysical tests and used an instrumented Virtual Egg to assess performance.ResultsResults obtained from 14 able-bodied subjects show that a classifier-based controller augmented with audio feedback enabled stronger internal model (p = 0.018) and better performance (p = 0.028) than a controller without this feedback.ConclusionsWe extended our previous work and accomplished the first steps on a path towards bridging the gap between research and clinical usability of a hand prosthesis. The main goal was to assess whether the ability to decouple internal model strength and motion variability using the continuous audio-augmented feedback extended to real-world use, where the inherent mechanical variability and dynamics in the mechanisms may contribute to a more complicated interplay between internal model formation and motion variability. We concluded that benefits of using audio-augmented feedback for improving internal model strength of myoelectric controllers extend beyond a virtual target acquisition task to include control of a prosthetic hand.

[1]  M. Keith,et al.  A neural interface provides long-term stable natural touch perception , 2014, Science Translational Medicine.

[2]  Christian Cipriani,et al.  Vibrotactile Sensory Substitution Elicits Feeling of Ownership of an Alien Hand , 2012, PloS one.

[3]  Nitish V. Thakor,et al.  Testing a Prosthetic Haptic Feedback Simulator With an Interactive Force Matching Task , 2008 .

[4]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[5]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[6]  Robert D. Lipschutz,et al.  The use of targeted muscle reinnervation for improved myoelectric prosthesis control in a bilateral shoulder disarticulation amputee , 2004, Prosthetics and orthotics international.

[7]  T. Matsushima,et al.  Striatal and Tegmental Neurons Code Critical Signals for Temporal-Difference Learning of State Value in Domestic Chicks , 2016, Front. Neurosci..

[8]  Steven S Hsiao,et al.  Sensory feedback for upper limb prostheses. , 2011, Progress in brain research.

[9]  Erik Scheme,et al.  Electromyogram pattern recognition for control of powered upper-limb prostheses: state of the art and challenges for clinical use. , 2011, Journal of rehabilitation research and development.

[10]  N. Hogan,et al.  Submovement changes characterize generalization of motor recovery after stroke , 2009, Cortex.

[11]  Christian Antfolk,et al.  Sensory feedback in upper limb prosthetics , 2013, Expert review of medical devices.

[12]  Dario Farina,et al.  GLIMPSE: Google Glass interface for sensory feedback in myoelectric hand prostheses , 2017, Journal of neural engineering.

[13]  Dario Farina,et al.  Electrotactile EMG feedback improves the control of prosthesis grasping force , 2016, Journal of neural engineering.

[14]  J. Houk,et al.  Deciding when and how to correct a movement: discrete submovements as a decision making process , 2007, Experimental Brain Research.

[15]  G A Clark,et al.  Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves , 2016, Journal of neural engineering.

[16]  Thierry Keller,et al.  Multichannel Electrotactile Feedback With Spatial and Mixed Coding for Closed-Loop Control of Grasping Force in Hand Prostheses , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[17]  Levi J. Hargrove,et al.  A Comparison of Surface and Intramuscular Myoelectric Signal Classification , 2007, IEEE Transactions on Biomedical Engineering.

[18]  D. Atkins,et al.  Epidemiologic Overview of Individuals with Upper-Limb Loss and Their Reported Research Priorities , 1996 .

[19]  Ahmed W. Shehata,et al.  Evaluating Internal Model Strength and Performance of Myoelectric Prosthesis Control Strategies , 2017, bioRxiv.

[20]  A. Bastian Understanding sensorimotor adaptation and learning for rehabilitation , 2008, Current opinion in neurology.

[21]  Luca Citi,et al.  Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses , 2014, Science Translational Medicine.

[22]  Yves G. Losier,et al.  A Bus-Based Smart Myoelectric Electrode/Amplifier—System Requirements , 2011, IEEE Transactions on Instrumentation and Measurement.

[23]  P. Lum,et al.  Internal models of upper limb prosthesis users when grasping and lifting a fragile object with their prosthetic limb , 2014, Experimental Brain Research.

[24]  Dario Farina,et al.  Closed-Loop Control of Grasping With a Myoelectric Hand Prosthesis: Which Are the Relevant Feedback Variables for Force Control? , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  Christian Cipriani,et al.  The SmartHand transradial prosthesis , 2011, Journal of NeuroEngineering and Rehabilitation.

[26]  K. J. Cole,et al.  Sensory-motor coordination during grasping and manipulative actions , 1992, Current Biology.

[27]  Wenwei Yu,et al.  Psycho-physiological assessment of a prosthetic hand sensory feedback system based on an auditory display: a preliminary study , 2012, Journal of NeuroEngineering and Rehabilitation.

[28]  Alexander W Dromerick,et al.  Feedforward control strategies of subjects with transradial amputation in planar reaching. , 2010, Journal of rehabilitation research and development.

[29]  G. Wood,et al.  Examining the Spatiotemporal Disruption to Gaze When Using a Myoelectric Prosthetic Hand , 2018, Journal of motor behavior.

[30]  C. Antfolk,et al.  Artificial Redirection of Sensation From Prosthetic Fingers to the Phantom Hand Map on Transradial Amputees: Vibrotactile Versus Mechanotactile Sensory Feedback , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[31]  N.V. Thakor,et al.  Towards the Control of Individual Fingers of a Prosthetic Hand Using Surface EMG Signals , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[32]  Keehoon Kim,et al.  Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees. , 2011, Brain : a journal of neurology.

[33]  Ahmed W. Shehata,et al.  The effect of myoelectric prosthesis control strategies and feedback level on adaptation rate for a target acquisition task , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).

[34]  I Delgado-Martínez,et al.  Fascicular nerve stimulation and recording using a novel double-aisle regenerative electrode , 2017, Journal of neural engineering.

[35]  Wenwei Yu,et al.  Human-Machine Interface for the Control of Multi-Function Systems Based on Electrocutaneous Menu: Application to Multi-Grasp Prosthetic Hands , 2015, PloS one.

[36]  Cara E. Stepp,et al.  Vibrotactile Sensory Substitution for Electromyographic Control of Object Manipulation , 2013, IEEE Transactions on Biomedical Engineering.

[37]  Jacqueline S. Hebert,et al.  Novel Targeted Sensory Reinnervation Technique to Restore Functional Hand Sensation After Transhumeral Amputation , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[38]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[39]  Dario Farina,et al.  EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis , 2015, Journal of NeuroEngineering and Rehabilitation.

[40]  V. Mathiowetz,et al.  Adult norms for the Box and Block Test of manual dexterity. , 1985, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[41]  Silvestro Micera,et al.  On the Shared Control of an EMG-Controlled Prosthetic Hand: Analysis of User–Prosthesis Interaction , 2008, IEEE Transactions on Robotics.

[42]  Adrian D. C. Chan,et al.  A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses , 2005, IEEE Transactions on Biomedical Engineering.

[43]  Dario Farina,et al.  User adaptation in Myoelectric Man-Machine Interfaces , 2017, Scientific Reports.

[44]  Thierry Keller,et al.  Short- and Long-Term Learning of Feedforward Control of a Myoelectric Prosthesis with Sensory Feedback by Amputees , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[45]  Joseph A. Doeringer,et al.  Intermittency in preplanned elbow movements persists in the absence of visual feedback. , 1998, Journal of neurophysiology.

[46]  Reva E. Johnson,et al.  Adaptation to random and systematic errors: Comparison of amputee and non-amputee control interfaces with varying levels of process noise , 2017, PloS one.

[47]  Hiroshi Imamizu,et al.  Human cerebellar activity reflecting an acquired internal model of a new tool , 2000, Nature.

[48]  Loredana Zollo,et al.  Literature Review on Needs of Upper Limb Prosthesis Users , 2016, Front. Neurosci..

[49]  W.J. Tompkins,et al.  Electrotactile and vibrotactile displays for sensory substitution systems , 1991, IEEE Transactions on Biomedical Engineering.

[50]  Andrew G. Barto,et al.  The Emergence of Multiple Movement Units in the Presence of Noise and Feedback Delay , 2001, NIPS.

[51]  Ahmed W Shehata,et al.  Audible Feedback Improves Internal Model Strength and Performance of Myoelectric Prosthesis Control , 2018, Scientific Reports.

[52]  Dudley S. Childress,et al.  Closed-loop control in prosthetic systems: Historical perspective , 2006, Annals of Biomedical Engineering.

[53]  J. Wheeler,et al.  Investigation of Rotational Skin Stretch for Proprioceptive Feedback With Application to Myoelectric Systems , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[54]  Hannes Bleuler,et al.  Brain Incorporation of Artificial Limbs and Role of Haptic Feedback , 2014 .

[55]  M. Zafar,et al.  Effectiveness of supplemental grasp-force feedback in the presence of vision , 2000, Medical and Biological Engineering and Computing.

[56]  Alicia J. Davis,et al.  Surveying the interest of individuals with upper limb loss in novel prosthetic control techniques , 2015, Journal of NeuroEngineering and Rehabilitation.

[57]  Christian Cipriani,et al.  Non-Invasive, Temporally Discrete Feedback of Object Contact and Release Improves Grasp Control of Closed-Loop Myoelectric Transradial Prostheses , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[58]  Luca Faes,et al.  Small-sample characterization of stochastic approximation staircases in forced-choice adaptive threshold estimation , 2007, Perception & psychophysics.

[59]  Jonathon W. Sensinger,et al.  A third arm — Design of a bypass prosthesis enabling incorporation , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).