A system for sensory motor rehabilitation of the upper limb with virtual reality, exoskeleton robot, and real objects

Technology assisted therapy has the potential to transform rehabilitation options available, and to dramatically increase the reach of today's healthcare system. Yet challenges persist in rendering translational application designs that optimize the full potential of technology and create value for the patient and the therapist. In a step towards optimizing value of technologies for practical applications to support very weak patients who might otherwise be unable to participate in traditional therapies, an integrated sensory motor training station was designed and developed. Inspired by recent neuroscientific research findings the goal of the design was to provide concurrent first person perspective immersive action observation of both virtual and real elements for motor and sensory experience; the system incorporates a virtual limb proxy that can be personalized and actuated by the robot and that is accompanied by exercise practice in peripersonal space for a plasticity promoting experience for the hand and arm. The station uses virtual reality and real objects for visual sensory experience, real objects also provide tactile sensory experience, and an exoskeleton upper limb robot provides assistance to patients. For many patients, successful movement and movement intensity required in rehabilitation is not achievable without the robot assistance. The multi-sensory features of the system promote a top-down strategy for training the upper limb (hand and arm) complementing the robot training; the system is ideally targeted for weak patients and those with tactile or proprioception sensory loss and those who are known to benefit from multi-sensory experiences.

[1]  Bharat B. Biswal,et al.  fMRI Analysis of Neural Mechanisms Underlying Rehabilitation in Virtual Reality: Activating Secondary Motor Areas , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[2]  P H Chappell,et al.  A robotic workstation for stroke rehabilitation of the upper extremity using FES. , 2009, Medical engineering & physics.

[3]  D. Mozaffarian,et al.  Heart disease and stroke statistics--2010 update: a report from the American Heart Association. , 2010, Circulation.

[4]  J. Jakobsen,et al.  Residual neuropathy in long-term population-based follow-up of Guillain–Barré syndrome , 2005, Neurology.

[5]  Gian Luigi Lenzi,et al.  Motor Recovery After Early Brain Damage A Case of Brain Plasticity , 1994, Stroke.

[6]  Craig S. Chapman,et al.  Obstacle avoidance during online corrections. , 2010, Journal of vision.

[7]  Aina Puce,et al.  Viewing the motion of human body parts activates different regions of premotor, temporal, and parietal cortex , 2004, NeuroImage.

[8]  J. Liepert,et al.  Training‐induced changes of motor cortex representations in stroke patients , 2000, Acta neurologica Scandinavica.

[9]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[10]  F. Pavani,et al.  Left tactile extinction following visual stimulation of a rubber hand. , 2000, Brain : a journal of neurology.

[11]  W. Penfield,et al.  SOMATIC MOTOR AND SENSORY REPRESENTATION IN THE CEREBRAL CORTEX OF MAN AS STUDIED BY ELECTRICAL STIMULATION , 1937 .

[12]  T. Elbert,et al.  New treatments in neurorehabiliation founded on basic research , 2002, Nature Reviews Neuroscience.

[13]  Hermano Igo Krebs,et al.  Therapeutic Robotics: A Technology Push: Stroke rehabilitation is being aided by robots that guide movement of shoulders and elbows, wrists, hands, arms and ankles to significantly improve recovery of patients. , 2006, Proceedings of the IEEE. Institute of Electrical and Electronics Engineers.

[14]  G. Wulf,et al.  Conceptions of Ability Affect Motor Learning , 2009, Journal of motor behavior.

[15]  M. Goodale,et al.  Visual pathways to perception and action. , 1993, Progress in brain research.

[16]  Maureen K. Holden,et al.  Virtual Environments for Motor Rehabilitation: Review , 2005, Cyberpsychology Behav. Soc. Netw..

[17]  F. Binkofski,et al.  The mirror neuron system and action recognition , 2004, Brain and Language.

[18]  Jeroen B. J. Smeets,et al.  Avoiding moving obstacles , 2008, Experimental Brain Research.

[19]  P. Haggard,et al.  Viewing the body modulates tactile receptive fields , 2007, Experimental Brain Research.

[20]  R. Nudo Remodeling of cortical motor representations after stroke: implications for recovery from brain damage , 1997, Molecular Psychiatry.

[21]  M. Iacoboni,et al.  The mirror neuron system and the consequences of its dysfunction , 2006, Nature Reviews Neuroscience.

[22]  E. Tunik,et al.  Sensorimotor training in virtual reality: a review. , 2009, NeuroRehabilitation.

[23]  A. Chiò,et al.  Guillain-Barré syndrome , 2003, Neurology.

[24]  Sabah Master,et al.  Task-specific increase in corticomotor excitability during tactile discrimination , 2009, Experimental Brain Research.

[25]  Robert Riener,et al.  ARMin: a robot for patient-cooperative arm therapy , 2007, Medical & Biological Engineering & Computing.

[26]  Vincent S. Huang,et al.  Robotic neurorehabilitation: a computational motor learning perspective , 2009, Journal of NeuroEngineering and Rehabilitation.

[27]  Y. Paulignan,et al.  An Interference Effect of Observed Biological Movement on Action , 2003, Current Biology.

[28]  Michael Recce,et al.  A virtual reality based exercise system for hand rehabilitation post-stroke: transfer to function , 2004, EMBC 2004.

[29]  Maria V. Sanchez-Vives,et al.  Virtual Hand Illusion Induced by Visuomotor Correlations , 2010, PloS one.

[30]  Douglas C Noll,et al.  Cortical Plasticity During Three-Week Motor Skill Learning , 2004, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[31]  Fary Khan,et al.  Rehabilitation in Guillian Barre syndrome. , 2004, Australian family physician.

[32]  J. Meulstee,et al.  Prognostic factors of Guillain-Barré syndrome after intravenous immunoglobulin or plasma exchange , 1999, Neurology.

[33]  Melvyn A. Goodale,et al.  Seeing all the obstacles in your way: the effect of visual feedback and visual feedback schedule on obstacle avoidance while reaching , 2010, Experimental Brain Research.

[34]  R. Johansson,et al.  Responses in glabrous skin mechanoreceptors during precision grip in humans , 2004, Experimental Brain Research.

[35]  M A Goodale,et al.  Kinematic analysis of limb movements in neuropsychological research: subtle deficits and recovery of function. , 1990, Canadian journal of psychology.

[36]  F L Mastaglia,et al.  Raised corticomotor excitability of M1 forearm area following anodal tDCS is sustained during robotic wrist therapy in chronic stroke. , 2009, Restorative neurology and neuroscience.

[37]  Peter Langhorne,et al.  Effects of Augmented Exercise Therapy Time After Stroke: A Meta-Analysis , 2004, Stroke.

[38]  David Whitney,et al.  The influence of visual motion on fast reaching movements to a stationary object , 2003, Nature.

[39]  C. Fisher,et al.  Concerning the Mechanism of Recovery in Stroke Hemiphegia , 1992, Canadian Journal of Neurological Sciences / Journal Canadien des Sciences Neurologiques.

[40]  J. Soechting,et al.  Somatosensory cortical activity in relation to arm posture: nonuniform spatial tuning. , 1996, Journal of neurophysiology.

[41]  G. Kwakkel,et al.  Understanding the pattern of functional recovery after stroke: facts and theories. , 2004, Restorative neurology and neuroscience.

[42]  Henk J Stam,et al.  Analysing the favourable effects of physical exercise: relationships between physical fitness, fatigue and functioning in Guillain-Barré syndrome and chronic inflammatory demyelinating polyneuropathy. , 2007, Journal of rehabilitation medicine.

[43]  J. Krakauer Motor learning: its relevance to stroke recovery and neurorehabilitation. , 2006, Current opinion in neurology.

[44]  Steven C Cramer,et al.  Repairing the human brain after stroke. II. Restorative therapies , 2008, Annals of neurology.

[45]  Angela R. Laird,et al.  ALE meta-analysis of action observation and imitation in the human brain , 2010, NeuroImage.

[46]  C. Spence,et al.  Visual Capture of Touch: Out-of-the-Body Experiences With Rubber Gloves , 2000, Psychological science.

[47]  P. Schmitz,et al.  Mild forms of Guillain-Barré syndrome in an epidemiologic survey in the Netherlands , 2000, Neurology.

[48]  Rajat Dhar,et al.  The morbidity and outcome of patients with Guillain–Barré syndrome admitted to the intensive care unit , 2008, Journal of the Neurological Sciences.

[49]  G. Kwakkel,et al.  The impact of physical therapy on functional outcomes after stroke: what's the evidence? , 2004, Clinical rehabilitation.

[50]  N. Hogan,et al.  Motions or muscles? Some behavioral factors underlying robotic assistance of motor recovery. , 2006, Journal of rehabilitation research and development.

[51]  Patrick Haggard,et al.  On-Line Control of Grasping Actions: Object-Specific Motor Facilitation Requires Sustained Visual Input , 2007, The Journal of Neuroscience.

[52]  L. Cohen,et al.  Concurrent action observation modulates practice‐induced motor memory formation , 2008, The European journal of neuroscience.

[53]  Catherine E Lang,et al.  Counting Repetitions: An Observational Study of Outpatient Therapy for People with Hemiparesis Post-Stroke , 2007, Journal of neurologic physical therapy : JNPT.

[54]  N. Ward Compensatory mechanisms in the aging motor system , 2006, Ageing Research Reviews.

[55]  D. Mozaffarian,et al.  Executive summary: heart disease and stroke statistics--2010 update: a report from the American Heart Association. , 2010, Circulation.

[56]  Jeremy D Wong,et al.  Visual cues signaling object grasp reduce interference in motor learning. , 2009, Journal of neurophysiology.

[57]  J. Feigenson,et al.  Factors Influencing Outcome and Length of Stay in a Stroke Rehabilitation Unit Part 1. Analysis of 248 Unscreened Patients ‐Medical and Functional Prognostic Indicators , 1977, Stroke.

[58]  Lynne Turner-Stokes,et al.  Multidisciplinary care for Guillain-Barré syndrome. , 2010, The Cochrane database of systematic reviews.

[59]  Dennis A. Nowak,et al.  The impact of stroke on the performance of grasping: Usefulness of kinetic and kinematic motion analysis , 2008, Neuroscience & Biobehavioral Reviews.

[60]  P. Raghavan,et al.  The nature of hand motor impairment after stroke and its treatment , 2007, Current treatment options in cardiovascular medicine.

[61]  G. Holmes,et al.  Sensory disturbances from cerebral lesions , 1911 .

[62]  Nicholas P. Holmes,et al.  The body schema and multisensory representation(s) of peripersonal space , 2004, Cognitive Processing.

[63]  G. Wulf,et al.  Social-comparative feedback affects motor skill learning. , 2010, Quarterly journal of experimental psychology.

[64]  C. Terwee,et al.  Effectiveness of exercise therapy: a best-evidence summary of systematic reviews. , 2005, The Australian journal of physiotherapy.

[65]  Bent Indredavik,et al.  Not All Stroke Units Are the Same: A Comparison of Physical Activity Patterns in Melbourne, Australia, and Trondheim, Norway , 2008, Stroke.

[66]  J. Feigenson,et al.  Factors Influencing Outcome and Length of Stay in a Stroke Rehabilitation Unit: Part 2. Comparison of 318 Screened and 248 Unscreened Patients , 1977, Stroke.

[67]  E. Burdet,et al.  Robot-assisted rehabilitation of hand function. , 2010, Current opinion in neurology.

[68]  M. Johnson,et al.  Quantifying kinematics of purposeful movements to real, imagined, or absent functional objects: Implications for modelling trajectories for robot-assisted ADL tasks** , 2007, Journal of NeuroEngineering and Rehabilitation.

[69]  J. Bussmann,et al.  Physical training and fatigue, fitness, and quality of life in Guillain–Barré syndrome and CIDP , 2004, Neurology.

[70]  R. Nudo,et al.  Neural Substrates for the Effects of Rehabilitative Training on Motor Recovery After Ischemic Infarct , 1996, Science.

[71]  Qinyin Qiu,et al.  Incorporating Haptic Effects Into Three-Dimensional Virtual Environments to Train the Hemiparetic Upper Extremity , 2009, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[72]  A Farnè,et al.  Dynamic size-change of peri-hand space through tool-use: spatial extension or shift of the multi-sensory area. , 2007, Journal of neuropsychology.

[73]  H. Krebs,et al.  Effects of Robot-Assisted Therapy on Upper Limb Recovery After Stroke: A Systematic Review , 2008, Neurorehabilitation and neural repair.

[74]  Julie Duque,et al.  Encoding a motor memory in the older adult by action observation , 2006, NeuroImage.

[75]  Grant D. Huang,et al.  Robot-assisted therapy for long-term upper-limb impairment after stroke. , 2010, The New England journal of medicine.

[76]  John A. Pyles,et al.  Neural adaptation for novel objects during dynamic articulation , 2009, Neuropsychologia.

[77]  L. Stone,et al.  Rehabilitation of hemiparesis after stroke with a mirror , 1999, The Lancet.

[78]  G. Rizzolatti,et al.  Neural Circuits Underlying Imitation Learning of Hand Actions An Event-Related fMRI Study , 2004, Neuron.

[79]  Qinyin Qiu,et al.  Design of a Virtual Reality-Based System For Hand and Arm Rehabilitation , 2007, 2007 IEEE 10th International Conference on Rehabilitation Robotics.

[80]  Eugene Tunik,et al.  Virtual reality to maximize function for hand and arm rehabilitation: exploration of neural mechanisms. , 2009, Studies in health technology and informatics.

[81]  J. Ryan,et al.  Cognitive strategy use to enhance motor skill acquisition post-stroke: A critical review , 2009, Brain injury.

[82]  H. Coslett,et al.  From maps to form to space: Touch and the body schema , 2010, Neuropsychologia.

[83]  P Tallal,et al.  Cortical plasticity underlying perceptual, motor, and cognitive skill development: implications for neurorehabilitation. , 1996, Cold Spring Harbor symposia on quantitative biology.

[84]  Leslie G. Ungerleider,et al.  Formation of a Motor Memory by Action Observation , 2005, The Journal of Neuroscience.

[85]  B. Dobkin Activity‐dependent learning contributes to motor recovery , 1998, Annals of neurology.

[86]  Charles Spence,et al.  Head orientation biases tactile localization , 2007, Brain Research.

[87]  A. Timmermans,et al.  Arm and hand skills: Training preferences after stroke , 2009, Disability and rehabilitation.

[88]  R. Hughes,et al.  Guillain-Barré syndrome , 1994, The Lancet.

[89]  J F Kalaska,et al.  Central neural mechanisms of touch and proprioception. , 1994, Canadian journal of physiology and pharmacology.

[90]  Christina T Fuentes,et al.  Where is your arm? Variations in proprioception across space and tasks. , 2010, Journal of neurophysiology.

[91]  G. Zeloni,et al.  Seeing or not seeing where your hands are , 2000, Experimental Brain Research.

[92]  J. Kalaska,et al.  Proprioceptive activity in primate primary somatosensory cortex during active arm reaching movements. , 1994, Journal of neurophysiology.

[93]  Ettore Beghi,et al.  Incidence and clinical features of acute inflammatory polyradiculoneuropathy in Lombardy, Italy, 1996 , 2004, Acta neurologica Scandinavica.

[94]  E. Tunik,et al.  A virtual reality-based system integrated with fmri to study neural mechanisms of action observation-execution: a proof of concept study. , 2009, Restorative neurology and neuroscience.

[95]  Alvaro Pascual-Leone,et al.  The time course of off-line motor sequence learning. , 2005, Brain research. Cognitive brain research.

[96]  J. Lackner,et al.  Fingertip contact influences human postural control , 2007, Experimental Brain Research.

[97]  M. Kossut,et al.  Learning‐induced expansion of cortical maps – what happens to adjacent cortical representations? , 1998, Neuroreport.

[98]  J. Kalaska,et al.  Parietal area 5 neuronal activity encodes movement kinematics, not movement dynamics , 2004, Experimental Brain Research.

[99]  Jaap H. Buurke,et al.  The Effectiveness of the Bobath Concept in Stroke Rehabilitation: What is the Evidence? , 2009, Stroke.

[100]  M. Goodale,et al.  Chapter 28 Visual pathways to perception and action , 1993 .

[101]  H. C. Dijkerman,et al.  Interference of grasping observation during prehension, a behavioural study , 2006, Experimental Brain Research.

[102]  G Rizzolatti,et al.  The Space Around Us , 1997, Science.

[103]  A. Timmermans,et al.  Technology-assisted training of arm-hand skills in stroke: concepts on reacquisition of motor control and therapist guidelines for rehabilitation technology design , 2009, Journal of NeuroEngineering and Rehabilitation.

[104]  Eliane C Magdalon,et al.  Virtual reality environments to enhance upper limb functional recovery in patients with hemiparesis. , 2009, Studies in health technology and informatics.

[105]  P. Schmitz,et al.  Fatigue in immune-mediated polyneuropathies , 1999, Neurology.

[106]  Geoffrey Donnan,et al.  Inactive and Alone: Physical Activity Within the First 14 Days of Acute Stroke Unit Care , 2004, Stroke.

[107]  KM Jacobs,et al.  Reshaping the cortical motor map by unmasking latent intracortical connections , 1991, Science.

[108]  N. Hogan,et al.  Robotic devices as therapeutic and diagnostic tools for stroke recovery. , 2009, Archives of neurology.

[109]  J. Tanji,et al.  Neuronal activity in the primate supplementary, pre-supplementary and premotor cortex during externally and internally instructed sequential movements , 1994, Neuroscience Research.

[110]  I. Robertson,et al.  Rehabilitation of brain damage: brain plasticity and principles of guided recovery. , 1999, Psychological bulletin.

[111]  Andrea Serino,et al.  Action-dependent plasticity in peripersonal space representations , 2008, Cognitive neuropsychology.

[112]  Anna Czlonkowska,et al.  Underfunding of Stroke Research: A Europe-Wide Problem , 2004, Stroke.

[113]  J. Mazziotta,et al.  Cortical mechanisms of human imitation. , 1999, Science.

[114]  D. Kiper,et al.  Observing Virtual Arms that You Imagine Are Yours Increases the Galvanic Skin Response to an Unexpected Threat , 2008, PloS one.

[115]  M. Goodale Transforming vision into action , 2011, Vision Research.

[116]  P. Schmitz,et al.  Fatigue in immune-mediated polyneuropathies. European Inflammatory Neuropathy Cause and Treatment (INCAT) Group. , 1999, Neurology.

[117]  H. Branch Coslett,et al.  Two-component models of reaching: Evidence from deafferentation in a Fitts’ law task , 2009, Neuroscience Letters.

[118]  P. Haggard,et al.  Can vision of the body ameliorate impaired somatosensory function? , 2007, Neuropsychologia.

[119]  S. Kitazawa,et al.  Reversal of subjective temporal order due to arm crossing , 2001, Nature Neuroscience.

[120]  C. Asanuma Mapping movements within a moving motor map , 1991, Trends in Neurosciences.

[121]  J. Lackner,et al.  Vestibular, proprioceptive, and haptic contributions to spatial orientation. , 2005, Annual review of psychology.

[122]  Fary Khan,et al.  Use of the International Classification of Functioning, Disability and Health to identify preliminary comprehensive and brief core sets for Guillain Barre syndrome , 2011, Disability and rehabilitation.

[123]  I. Kaufman The Cerebral Cortex of Man: A Clinical Study of Localization of Function , 1951 .

[124]  D. Reinkensmeyer,et al.  Review of control strategies for robotic movement training after neurologic injury , 2009, Journal of NeuroEngineering and Rehabilitation.

[125]  Elisabetta Làdavas,et al.  Multisensory‐based Approach to the Recovery of Unisensory Deficit , 2008, Annals of the New York Academy of Sciences.

[126]  S. Brochard,et al.  What's new in new technologies for upper extremity rehabilitation? , 2010, Current opinion in neurology.

[127]  Glyn W. Humphreys,et al.  Motor facilitation following action observation: A behavioural study in prehensile action , 2003, Brain and Cognition.

[128]  F. Binkofski,et al.  Motor functions of the Broca’s region , 2004, Brain and Language.

[129]  D. Wegner,et al.  Modulating the sense of agency with external cues , 2009, Consciousness and Cognition.

[130]  P. Haggard,et al.  Touch and the body , 2010, Neuroscience & Biobehavioral Reviews.

[131]  Andrew A G Mattar,et al.  Motor Learning by Observing , 2005, Neuron.

[132]  Hermano Igo Krebs,et al.  Therapeutic Robotics: A Technology Push , 2006, Proceedings of the IEEE.

[133]  Ilja Frissen,et al.  Visual recalibration of auditory spatial perception: two separate neural circuits for perceptual learning , 2009, The European journal of neuroscience.

[134]  M. Hallett,et al.  Improving hand function in chronic stroke. , 2002, Archives of neurology.

[135]  Ajitkumar P. Mulavara,et al.  The interplay between strategic and adaptive control mechanisms in plastic recalibration of locomotor function , 2007, Experimental Brain Research.

[136]  M. Levin,et al.  Virtual Reality in Stroke Rehabilitation: A Systematic Review of its Effectiveness for Upper Limb Motor Recovery , 2007, Topics in stroke rehabilitation.

[137]  C. Spence,et al.  Confusing the mind by crossing the hands. , 2002, Brain research. Cognitive brain research.

[138]  J H Rees,et al.  Epidemiological study of Guillain-Barré syndrome in south east England , 1998, Journal of neurology, neurosurgery, and psychiatry.

[139]  J. Liepert,et al.  Treatment-induced cortical reorganization after stroke in humans. , 2000, Stroke.

[140]  Gerard G. Fluet,et al.  Recovery of hand function in virtual reality: Training hemiparetic hand and arm together or separately , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[141]  Qinyin Qiu,et al.  Journal of Neuroengineering and Rehabilitation Design of a Complex Virtual Reality Simulation to Train Finger Motion for Persons with Hemiparesis: a Proof of Concept Study , 2022 .

[142]  Lara A. Boyd,et al.  Implicit motor-sequence learning in humans following unilateral stroke: the impact of practice and explicit knowledge , 2001, Neuroscience Letters.

[143]  Richard J Barohn,et al.  Supportive care for patients with Guillain-Barré syndrome. , 2005, Archives of neurology.

[144]  M. Guadagnoli,et al.  Challenge Point: A Framework for Conceptualizing the Effects of Various Practice Conditions in Motor Learning , 2004, Journal of motor behavior.

[145]  Prats Viñas Jm,et al.  Guillain–Barré Syndrome , 1992, Pediatric Practice Guidelines.

[146]  J. Lackner,et al.  Stabilization of posture by precision contact of the index finger. , 1994, Journal of vestibular research : equilibrium & orientation.

[147]  Andrew N. Meltzoff,et al.  Neural circuits involved in imitation and perspective-taking , 2006, NeuroImage.

[148]  Melvyn A Goodale,et al.  Dual routes to action: contributions of the dorsal and ventral streams to adaptive behavior. , 2005, Progress in brain research.

[149]  Fisher Cm,et al.  Concerning the Mechanism of Recovery in Stroke Hemiphegia , 1992 .

[150]  N. Byl,et al.  Effectiveness of Sensory and Motor Rehabilitation of the Upper Limb Following the Principles of Neuroplasticity: Patients Stable Poststroke , 2003, Neurorehabilitation and neural repair.

[151]  M. Jeannerod,et al.  The coupling of arm and finger movements during prehension , 2004, Experimental Brain Research.