When Less Is More – Discrete Tactile Feedback Dominates Continuous Audio Biofeedback in the Integrated Percept While Controlling a Myoelectric Prosthetic Hand

State of the art myoelectric hand prostheses can restore some feedforward motor function to their users, but they cannot yet restore sensory feedback. It has been shown, using psychophysical tests, that multi-modal sensory feedback is readily used in the formation of the users’ representation of the control task in their central nervous system – their internal model. Hence, to fully describe the effect of providing feedback to prosthesis users, not only should functional outcomes be assessed, but so should the internal model. In this study, we compare the complex interactions between two different feedback types, as well as a combination of the two, on the internal model, and the functional performance of naïve participants without limb difference. We show that adding complementary audio biofeedback to visual feedback enables the development of a significantly stronger internal model for controlling a myoelectric hand compared to visual feedback alone, but adding discrete vibrotactile feedback to vision does not. Both types of feedback, however, improved the functional grasping abilities to a similar degree. Contrary to our expectations, when both types of feedback are combined, the discrete vibrotactile feedback seems to dominate the continuous audio feedback. This finding indicates that simply adding sensory information may not necessarily enhance the formation of the internal model in the short term. In fact, it could even degrade it. These results support our argument that assessment of the internal model is crucial to understanding the effects of any type of feedback, although we cannot be sure that the metrics used here describe the internal model exhaustively. Furthermore, all the feedback types tested herein have been proven to provide significant functional benefits to the participants using a myoelectrically controlled robotic hand. This article, therefore, proposes a crucial conceptual and methodological addition to the evaluation of sensory feedback for upper limb prostheses – the internal model – as well as new types of feedback that promise to significantly and considerably improve functional prosthesis control.

[1]  Dario Farina,et al.  EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis , 2015, Journal of NeuroEngineering and Rehabilitation.

[2]  Ahmed W Shehata,et al.  Audible Feedback Improves Internal Model Strength and Performance of Myoelectric Prosthesis Control , 2018, Scientific Reports.

[3]  Ahmed W. Shehata,et al.  The effect of myoelectric prosthesis control strategies and feedback level on adaptation rate for a target acquisition task , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).

[4]  Dario Farina,et al.  Myocontrol is closed-loop control: incidental feedback is sufficient for scaling the prosthesis force in routine grasping , 2018, Journal of NeuroEngineering and Rehabilitation.

[5]  Dudley S. Childress,et al.  Closed-loop control in prosthetic systems: Historical perspective , 2006, Annals of Biomedical Engineering.

[6]  Simona Crea,et al.  Providing Time-Discrete Gait Information by Wearable Feedback Apparatus for Lower-Limb Amputees: Usability and Functional Validation , 2015, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[7]  N. Hogan,et al.  Submovement changes characterize generalization of motor recovery after stroke , 2009, Cortex.

[8]  Christian Cipriani,et al.  Discrete Vibro-Tactile Feedback Prevents Object Slippage in Hand Prostheses More Intuitively Than Other Modalities , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[9]  Dario Farina,et al.  Building an internal model of a myoelectric prosthesis via closed-loop control for consistent and routine grasping , 2015, Experimental Brain Research.

[10]  Dario Farina,et al.  GLIMPSE: Google Glass interface for sensory feedback in myoelectric hand prostheses , 2017, Journal of neural engineering.

[11]  J. Houk,et al.  Deciding when and how to correct a movement: discrete submovements as a decision making process , 2007, Experimental Brain Research.

[12]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[13]  Nitish V. Thakor,et al.  Testing a Prosthetic Haptic Feedback Simulator With an Interactive Force Matching Task , 2008 .

[14]  Christian Cipriani,et al.  A cosmetic prosthetic digit with bioinspired embedded touch feedback , 2017, 2017 International Conference on Rehabilitation Robotics (ICORR).

[15]  V. Mathiowetz,et al.  Adult norms for the Box and Block Test of manual dexterity. , 1985, The American journal of occupational therapy : official publication of the American Occupational Therapy Association.

[16]  Christian Antfolk,et al.  Sensory feedback in upper limb prosthetics , 2013, Expert review of medical devices.

[17]  Christian Cipriani,et al.  Improving internal model strength and performance of prosthetic hands using augmented feedback , 2018, Journal of NeuroEngineering and Rehabilitation.

[18]  K. J. Cole,et al.  Sensory-motor coordination during grasping and manipulative actions , 1992, Current Biology.

[19]  David C Knill,et al.  Mixture models and the probabilistic structure of depth cues , 2003, Vision Research.

[20]  Joseph A. Doeringer,et al.  Intermittency in preplanned elbow movements persists in the absence of visual feedback. , 1998, Journal of neurophysiology.

[21]  Thierry Keller,et al.  Multichannel Electrotactile Feedback With Spatial and Mixed Coding for Closed-Loop Control of Grasping Force in Hand Prostheses , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[22]  Reva E. Johnson,et al.  Adaptation to random and systematic errors: Comparison of amputee and non-amputee control interfaces with varying levels of process noise , 2017, PloS one.

[23]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[24]  Hiroshi Imamizu,et al.  Human cerebellar activity reflecting an acquired internal model of a new tool , 2000, Nature.

[25]  A. Bastian Understanding sensorimotor adaptation and learning for rehabilitation , 2008, Current opinion in neurology.

[26]  D. Nowak,et al.  Preserved and Impaired Aspects of Feed-Forward Grip Force Control After Chronic Somatosensory Deafferentation , 2008, Neurorehabilitation and neural repair.

[27]  Konrad Paul Kording,et al.  Relevance of error: what drives motor adaptation? , 2009, Journal of neurophysiology.

[28]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[29]  R R Riso,et al.  Strategies for providing upper extremity amputees with tactile and hand position feedback--moving closer to the bionic arm. , 1999, Technology and health care : official journal of the European Society for Engineering and Medicine.

[30]  Benoit P. Delhaye,et al.  The neural basis of perceived intensity in natural and artificial touch , 2016, Science Translational Medicine.

[31]  Ahmed W. Shehata,et al.  Evaluating Internal Model Strength and Performance of Myoelectric Prosthesis Control Strategies , 2017, bioRxiv.

[32]  Alicia J. Davis,et al.  Surveying the interest of individuals with upper limb loss in novel prosthetic control techniques , 2015, Journal of NeuroEngineering and Rehabilitation.

[33]  Christian Cipriani,et al.  Non-Invasive, Temporally Discrete Feedback of Object Contact and Release Improves Grasp Control of Closed-Loop Myoelectric Transradial Prostheses , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[34]  Marc O. Ernst,et al.  A Bayesian view on multimodal cue integration , 2006 .

[35]  Ahmed W. Shehata,et al.  Mechanotactile Sensory Feedback Improves Embodiment of a Prosthetic Hand During Active Use , 2020, Frontiers in Neuroscience.

[36]  E. Bizzi,et al.  Controlling multijoint motor behavior. , 1987, Exercise and sport sciences reviews.

[37]  D. Elliott,et al.  The Utilization of Visual Feedback Information during Rapid Pointing Movements , 1985, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[38]  Purushothaman Geethanjali,et al.  Myoelectric control of prosthetic hands: state-of-the-art review , 2016, Medical devices.

[39]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[40]  S. Vijayakumar,et al.  The role of feed-forward and feedback processes for closed-loop prosthesis control , 2011, Journal of NeuroEngineering and Rehabilitation.

[41]  Luca Citi,et al.  Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses , 2014, Science Translational Medicine.

[42]  P. Lum,et al.  Internal models of upper limb prosthesis users when grasping and lifting a fragile object with their prosthetic limb , 2014, Experimental Brain Research.

[43]  Dario Farina,et al.  Closed-Loop Control of Grasping With a Myoelectric Hand Prosthesis: Which Are the Relevant Feedback Variables for Force Control? , 2014, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[44]  Dario Farina,et al.  The clinical relevance of advanced artificial feedback in the control of a multi-functional myoelectric prosthesis , 2018, Journal of NeuroEngineering and Rehabilitation.

[45]  H. Forssberg,et al.  Lighter or Heavier Than Predicted: Neural Correlates of Corrective Mechanisms during Erroneously Programmed Lifts , 2006, The Journal of Neuroscience.

[46]  Oonagh M. Giggins,et al.  Biofeedback in rehabilitation , 2013, Journal of NeuroEngineering and Rehabilitation.

[47]  M. Keith,et al.  A neural interface provides long-term stable natural touch perception , 2014, Science Translational Medicine.

[48]  R. Johansson,et al.  Predictive feed-forward sensory control during grasping and manipulation in man , 1993 .

[49]  Jorge Pomares,et al.  Evaluation of Optimal Vibrotactile Feedback for Force-Controlled Upper Limb Myoelectric Prostheses , 2019, Sensors.

[50]  Christian Cipriani,et al.  Humans can integrate feedback of discrete events in their sensorimotor control of a robotic hand , 2014, Experimental Brain Research.

[51]  Andrew G. Barto,et al.  The Emergence of Multiple Movement Units in the Presence of Noise and Feedback Delay , 2001, NIPS.