Illusory movement perception improves motor control for prosthetic hands

A perceptual illusion provides the sensation of complex bionic hand movements to human amputees, allowing real-time movement control without the necessity of vision. Good vibrations for movement perception The ability to sense the spatial position and movements of one’s own body (kinesthetic sense) is critical for limb use. Because prostheses do not provide physical feedback during movement, amputees may not feel that they are in control of their bodily movements (sense of agency) when manipulating a prosthesis. Marasco et al. developed an automated neural-machine interface that vibrates the muscles used for control of prosthetic hands. This system instilled kinesthetic sense in amputees, allowing them to control prosthetic hand movements in the absence of visual feedback and increasing their sense of agency. This approach might be an effective strategy for improving motor performance and quality of life in amputees. To effortlessly complete an intentional movement, the brain needs feedback from the body regarding the movement’s progress. This largely nonconscious kinesthetic sense helps the brain to learn relationships between motor commands and outcomes to correct movement errors. Prosthetic systems for restoring function have predominantly focused on controlling motorized joint movement. Without the kinesthetic sense, however, these devices do not become intuitively controllable. We report a method for endowing human amputees with a kinesthetic perception of dexterous robotic hands. Vibrating the muscles used for prosthetic control via a neural-machine interface produced the illusory perception of complex grip movements. Within minutes, three amputees integrated this kinesthetic feedback and improved movement control. Combining intent, kinesthesia, and vision instilled participants with a sense of agency over the robotic movements. This feedback approach for closed-loop control opens a pathway to seamless integration of minds and machines.

[1]  Courtney E. Shell,et al.  The neural response properties and cortical organization of a rapidly adapting muscle sensory group response that overlaps with the frequencies that elicit the kinesthetic illusion , 2017, PloS one.

[2]  H. Ehrsson,et al.  Moving a Rubber Hand that Feels Like Your Own: A Dissociation of Ownership and Agency , 2012, Front. Hum. Neurosci..

[3]  R. J. Beers,et al.  Motor Learning Is Optimally Tuned to the Properties of Motor Noise , 2009, Neuron.

[4]  O. Blanke,et al.  Changing motor perception by sensorimotor conflicts and body ownership , 2016, Scientific Reports.

[5]  P. Rossini,et al.  Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans , 2016, eLife.

[6]  E. Naito,et al.  Visuokinesthetic Perception of Hand Movement is Mediated by Cerebro–Cerebellar Interaction between the Left Cerebellum and Right Parietal Cortex , 2008, Cerebral cortex.

[7]  Nava Rubin,et al.  Bi-stable depth ordering of superimposed moving gratings. , 2008, Journal of vision.

[8]  B. Edin,et al.  Skin strain patterns provide kinaesthetic information to the human central nervous system. , 1995, The Journal of physiology.

[9]  Konrad P. Körding,et al.  Uncertainty of Feedback and State Estimation Determines the Speed of Motor Adaptation , 2009, Front. Comput. Neurosci..

[10]  Luca Faes,et al.  Small-sample characterization of stochastic approximation staircases in forced-choice adaptive threshold estimation , 2007, Perception & psychophysics.

[11]  Robert D. Lipschutz,et al.  Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: a case study , 2007, The Lancet.

[12]  Dominic Thewlis,et al.  Assessing Proprioception , 2015, Neurorehabilitation and neural repair.

[13]  Sukhvinder S. Obhi,et al.  Sense of agency in joint action: influence of human and computer co-actors , 2011, Experimental Brain Research.

[14]  Ann L. Edwards,et al.  Adaptive and Autonomous Switching: Shared Control of Powered Prosthetic Arms Using Reinforcement Learning , 2016 .

[15]  Todd A. Kuiken,et al.  The hyper-reinnervation of rat skeletal muscle , 1995, Brain Research.

[16]  Jonathan D. Cohen,et al.  Rubber hands ‘feel’ touch that eyes see , 1998, Nature.

[17]  Stephen T. Foldes,et al.  Intracortical microstimulation of human somatosensory cortex , 2016, Science Translational Medicine.

[18]  C Fraser,et al.  A case study of reaching by a user of a manually-operated artificial hand , 1981, Prosthetics and orthotics international.

[19]  Lauren H Smith,et al.  A comparison of the real-time controllability of pattern recognition to conventional myoelectric control for discrete and simultaneous movements , 2012, Journal of NeuroEngineering and Rehabilitation.

[20]  Luca Citi,et al.  Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses , 2014, Science Translational Medicine.

[21]  Lee Re Reassessing myoelectric control: is it time to look at alternatives? , 1987 .

[22]  U. Castiello The neuroscience of grasping , 2005, Nature Reviews Neuroscience.

[23]  N. A. Bernshteĭn The co-ordination and regulation of movements , 1967 .

[24]  D. Wolpert,et al.  Abnormalities in the awareness and control of action. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[25]  Reva E. Johnson,et al.  Adaptation to random and systematic errors: Comparison of amputee and non-amputee control interfaces with varying levels of process noise , 2017, PloS one.

[26]  Jacqueline S. Hebert,et al.  Updates in Targeted Sensory Reinnervation for Upper Limb Amputation , 2014, Current Surgery Reports.

[27]  G. Eklund,et al.  Normal variability of tonic vibration reflexes in man. , 1966, Experimental neurology.

[28]  W. Rymer,et al.  Joint position sense: the effects of muscle contraction. , 1980, Brain : a journal of neurology.

[29]  A. Newen,et al.  I move, therefore I am: A new theoretical framework to investigate agency and ownership , 2008, Consciousness and Cognition.

[30]  Benoit P. Delhaye,et al.  Key considerations in designing a somatosensory neuroprosthesis , 2016, Journal of Physiology-Paris.

[31]  Mohammad M D Sobuh,et al.  Visuomotor behaviours when using a myoelectric prosthesis , 2014, Journal of NeuroEngineering and Rehabilitation.

[32]  J. Lackner Some proprioceptive influences on the perceptual representation of body shape and orientation. , 1988, Brain : a journal of neurology.

[33]  Joseph E O'Doherty,et al.  A learning–based approach to artificial sensory feedback leads to optimal integration , 2014, Nature Neuroscience.

[34]  D. McCloskey,et al.  The contribution of muscle afferents to kinaesthesia shown by vibration induced illusions of movement and by the effects of paralysing joint afferents. , 1972, Brain : a journal of neurology.

[35]  Blair A. Lock,et al.  Redirection of cutaneous sensation from the hand to the chest skin of human amputees with targeted reinnervation , 2007, Proceedings of the National Academy of Sciences.

[36]  K. Horch,et al.  Object Discrimination With an Artificial Hand Using Electrical Stimulation of Peripheral Tactile and Proprioceptive Pathways With Intrafascicular Electrodes , 2011, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[38]  G.S. Dhillon,et al.  Direct neural sensory feedback and control of a prosthetic arm , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Axel Cleeremans,et al.  The relationship between human agency and embodiment , 2015, Consciousness and Cognition.

[40]  David J. Warren,et al.  Restoration of motor control and proprioceptive and cutaneous sensation in humans with prior upper-limb amputation via multiple Utah Slanted Electrode Arrays (USEAs) implanted in residual peripheral arm nerves , 2017, Journal of NeuroEngineering and Rehabilitation.

[41]  K. Horch,et al.  Effects of short-term training on sensory and motor function in severed nerves of long-term human amputees. , 2005, Journal of neurophysiology.

[42]  S. Gallagher Philosophical conceptions of the self: implications for cognitive science , 2000, Trends in Cognitive Sciences.

[43]  J. Roll,et al.  Kinaesthetic role of muscle afferents in man, studied by tendon vibration and microneurography , 2004, Experimental Brain Research.

[44]  M. Ernst,et al.  The statistical determinants of adaptation rate in human reaching. , 2008, Journal of vision.

[45]  Daniel Tan,et al.  Sensory feedback by peripheral nerve stimulation improves task performance in individuals with upper limb loss using a myoelectric prosthesis , 2016, Journal of neural engineering.

[46]  Danica Kragic,et al.  The GRASP Taxonomy of Human Grasp Types , 2016, IEEE Transactions on Human-Machine Systems.

[47]  P. Haggard,et al.  Voluntary action and conscious awareness , 2002, Nature Neuroscience.

[48]  T. Kuiken,et al.  Sensory capacity of reinnervated skin after redirection of amputated upper limb nerves to the chest , 2009, Brain : a journal of neurology.

[49]  A B Vallbo,et al.  Directional tuning of human forearm muscle afferents during voluntary wrist movements , 2001, The Journal of physiology.

[50]  Christian Ethier,et al.  On the functional organization and operational principles of the motor cortex , 2013, Front. Neural Circuits.

[51]  B Craske,et al.  Perception of impossible limb positions induced by tendon vibration. , 1977, Science.

[52]  Dustin J. Tyler,et al.  Stability and selectivity of a chronic, multi-contact cuff electrode for sensory stimulation in a human amputee , 2013 .

[53]  K O Johnson,et al.  Sensory discrimination: decision process. , 1980, Journal of neurophysiology.

[54]  M. Keith,et al.  A neural interface provides long-term stable natural touch perception , 2014, Science Translational Medicine.

[55]  Konrad Paul Kording,et al.  Estimating the sources of motor errors for adaptation and generalization , 2008, Nature Neuroscience.

[56]  M. Jeannerod,et al.  Sense of body and sense of action both contribute to self-recognition , 2002, Cognition.

[57]  James W. Moore,et al.  What Is the Sense of Agency and Why Does it Matter? , 2016, Front. Psychol..

[58]  D. McCloskey Differences between the senses of movement and position shown by the effects of loading and vibration of muscles in man. , 1973, Brain research.

[59]  K. Vogeley,et al.  The “sense of agency” and its underlying cognitive and neural mechanisms , 2008, Consciousness and Cognition.

[60]  Chloé Thyrion,et al.  Predicting any arm movement feedback to induce three-dimensional illusory movements in humans. , 2010, Journal of neurophysiology.

[61]  Keehoon Kim,et al.  Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees. , 2011, Brain : a journal of neurology.

[62]  Jacqueline S. Hebert,et al.  Characterizing the effects of amplitude, frequency and limb position on vibration induced movement illusions: Implications in sensory-motor rehabilitation. , 2015, Technology and health care : official journal of the European Society for Engineering and Medicine.

[63]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[64]  Dick H. Plettenburg,et al.  Bernstein’s Levels of Construction of Movements Applied to Upper Limb Prosthetics , 2012 .

[65]  Lee E Miller,et al.  Toward a Proprioceptive Neural Interface that Mimics Natural Cortical Activity. , 2016, Advances in experimental medicine and biology.

[66]  L. Wheaton Neurorehabilitation in upper limb amputation: understanding how neurophysiological changes can affect functional rehabilitation , 2017, Journal of NeuroEngineering and Rehabilitation.

[67]  S. Gandevia,et al.  Cutaneous receptors contribute to kinesthesia at the index finger, elbow, and knee. , 2005, Journal of neurophysiology.

[68]  F. Clippinger,et al.  A sensory feedback system for an upper-limb amputation prosthesis. , 1974, Bulletin of prosthetics research.

[69]  S. Gandevia,et al.  The proprioceptive senses: their roles in signaling body shape, body position and movement, and muscle force. , 2012, Physiological reviews.

[70]  U. Proske,et al.  Digital Object Identifier (DOI) 10.1007/s002210000380 RESEARCH ARTICLE , 2022 .

[71]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[72]  J. Bresciani,et al.  Trying to Move Your Unseen Static Arm Modulates Visually-Evoked Kinesthetic Illusion , 2013, PloS one.

[73]  Vikash Kumar,et al.  MuJoCo HAPTIX: A virtual reality system for hand manipulation , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[74]  Robert D. Lipschutz,et al.  Targeted muscle reinnervation for real-time myoelectric control of multifunction artificial arms. , 2009, JAMA.