Serious Games Are Not Serious Enough for Myoelectric Prosthetics

Serious games show a lot of potential for use in movement rehabilitation (eg, after a stroke, injury to the spinal cord, or limb loss). However, the nature of this research leads to diversity both in the background of the researchers and in the approaches of their investigation. Our close examination and categorization of virtual training software for upper limb prosthetic rehabilitation found that researchers typically followed one of two broad approaches: (1) focusing on the game design aspects to increase engagement and muscle training and (2) concentrating on an accurate representation of prosthetic training tasks, to induce task-specific skill transfer. Previous studies indicate muscle training alone does not lead to improved prosthetic control without a transfer-enabling task structure. However, the literature shows a recent surge in the number of game-based prosthetic training tools, which focus on engagement without heeding the importance of skill transfer. This influx appears to have been strongly influenced by the availability of both software and hardware, specifically the launch of a commercially available acquisition device and freely available high-profile game development engines. In this Viewpoint, we share our perspective on the current trends and progress of serious games for prosthetic training.

[1]  D. F. Lovely,et al.  A computer-aided myoelectric training system for young upper limb amputees , 1990 .

[2]  E. L. Morin,et al.  A myoelectric control evaluation and trainer system , 1994 .

[3]  Christian Balkenius,et al.  Neural Control of a Virtual Prosthesis , 1998 .

[4]  C. Light,et al.  Establishing a standardized clinical assessment tool of pathologic and prosthetic hand function: normative data, reliability, and validity. , 2002, Archives of physical medicine and rehabilitation.

[5]  Alcimar Soares,et al.  The Development of a Virtual Myoelectric Prosthesis Controlled by an EMG Pattern Recognition System Based on Neural Networks , 2004, Journal of Intelligent Information Systems.

[6]  José Luis Pons Rovira,et al.  Virtual reality training and EMG control of the MANUS hand prosthesis , 2005, Robotica.

[7]  Lance Chun Che Fung,et al.  Distinguishing games and simulation games from simulators , 2006, CIE.

[8]  B. Hudgins,et al.  A Real-Time Pattern Recognition Based Myoelectric Control Usability Study Implemented in a Virtual Environment , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  T. Takeuchi,et al.  A Training System for Myoelectric Prosthetic Hand in Virtual Environment , 2007, 2007 IEEE/ICME International Conference on Complex Medical Engineering.

[10]  R.J. Vogelstein,et al.  Air-Guitar Hero: A real-time video game interface for training and evaluation of dexterous upper-extremity neuroprosthetic control algorithms , 2008, 2008 IEEE Biomedical Circuits and Systems Conference.

[11]  R. de la Rosa,et al.  Myo-Pong: A neuromuscular game for the UVa-Neuromuscular Training System platform , 2008, 2008 Virtual Rehabilitation.

[12]  Alonso Alonso,et al.  The UVa-Neuromuscular Training System Platform , 2009, IWANN.

[13]  Adel Al-Jumaily,et al.  Electromyogram (EMG) driven system based virtual reality for prosthetic and rehabilitation devices , 2009, iiWAS.

[14]  R. Jacob Vogelstein,et al.  WiiEMG: A real-time environment for control of the Wii with surface electromyography , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[15]  Robert F Kirsch,et al.  Virtual Reality Environment for Simulating Tasks With a Myoelectric Prosthesis: An Assessment and Training Tool , 2011, Journal of prosthetics and orthotics : JPO.

[16]  Linda Resnik,et al.  Using virtual reality environment to facilitate training with advanced upper-limb prosthesis. , 2011, Journal of rehabilitation research and development.

[17]  Todd A Kuiken,et al.  Target Achievement Control Test: evaluating real-time myoelectric pattern-recognition control of multifunctional upper-limb prostheses. , 2011, Journal of rehabilitation research and development.

[18]  Rahman Davoodi,et al.  MSMS Software for VR Simulations of Neural Prostheses and Patient Training and Rehabilitation , 2011, MMVR.

[19]  Paolo Gardois,et al.  Implementation of Web 2.0 services in academic, medical and research libraries: a scoping review. , 2012, Health information and libraries journal.

[20]  Farbod Fahimi,et al.  The Development of a Myoelectric Training Tool for Above-Elbow Amputees , 2012, The open biomedical engineering journal.

[21]  Rahman Davoodi,et al.  Development of a Physics-Based Target Shooting Game to Train Amputee Users of Multijoint Upper Limb Prostheses , 2012, PRESENCE: Teleoperators and Virtual Environments.

[22]  Natalia Lopez,et al.  VIRTUAL REALITY UPPER LIMB MODEL CONTROLLED BY EMG SIGNALS , 2013 .

[23]  Alex,et al.  Using Augmented Reality Techniques to Simulate Myoelectric Upper Limb Prostheses , 2013 .

[24]  H. V. D. Van der Loos,et al.  Video Games and Rehabilitation: Using Design Principles to Enhance Engagement in Physical Therapy , 2013, Journal of neurologic physical therapy : JNPT.

[25]  Toshio Tsuji,et al.  A training system for the MyoBock hand in a virtual reality environment , 2013, 2013 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[26]  G. Bo Serious games en médecine et santé (SeGaMed 2013) : The REHAB@HOME Project: Engaging Game-based Home Rehabilitation for Improved Quality of Life. , 2013 .

[27]  Walter F. Bischof,et al.  Augmented reality improves myoelectric prosthesis training , 2014 .

[28]  Osamu Matsumoto,et al.  Hand posture and gesture recognition using MYO armband and spectral collaborative representation based classification , 2015, 2015 IEEE 4th Global Conference on Consumer Electronics (GCCE).

[29]  Chris Roast,et al.  Exploring virtual reality and prosthetic training , 2015, 2015 IEEE Virtual Reality (VR).

[30]  Naznin Virji-Babul,et al.  Therapists’ Perceptions of Social Media and Video Game Technologies in Upper Limb Rehabilitation , 2015, JMIR serious games.

[31]  Raoul M. Bongers,et al.  Virtual Training of the Myosignal , 2015, PloS one.

[32]  Theocharis Kyriacou,et al.  Feasibility of using combined EMG and kinematic signals for prosthesis control: A simulation study using a virtual reality environment , 2016, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[33]  Raoul M. Bongers,et al.  Learning an EMG Controlled Game: Task-Specific Adaptations and Transfer , 2016, PloS one.

[34]  Dario Farina,et al.  A novel immersive augmented reality system for prosthesis training and assessment , 2016, 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI).

[35]  Toshio Tsuji,et al.  A High-fidelity Virtual Training System for Myoelectric Prostheses Using an Immersive HMD , 2016 .

[36]  Raoul M. Bongers,et al.  Task-Oriented Gaming for Transfer to Prosthesis Use , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  Aaron Tabor,et al.  The Falling of Momo: A Myo-Electric Controlled Game to Support Research in Prosthesis Training , 2016, CHI PLAY.

[38]  Fares Kayali,et al.  Increasing motivation, effort and performance through game-based rehabilitation for upper limb myoelectric prosthesis control , 2017, 2017 International Conference on Virtual Rehabilitation (ICVR).

[39]  David R. Flatla,et al.  Designing Game-Based Myoelectric Prosthesis Training , 2017, CHI.

[40]  Bruno Bonnechere Serious Games in Physical Rehabilitation: From Theory to Practice , 2017 .

[41]  Toshio Tsuji,et al.  A virtual myoelectric prosthesis training system capable of providing instructions on hand operations , 2017 .

[42]  Peter Purgathofer,et al.  Game-Based Rehabilitation for Myoelectric Prosthesis Control , 2017, JMIR serious games.

[43]  Fares Kayali,et al.  Recommendations for Games to Increase Patient Motivation During Upper Limb Amputee Rehabilitation , 2017 .

[44]  Erik Scheme,et al.  Evaluation of Myoelectric Control Learning Using Multi-Session Game-Based Training , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[45]  Peter A. Smith,et al.  Usability testing games for prosthetic training , 2018, 2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH).

[46]  Peter A. Smith,et al.  The Impact of a Custom Electromyograph (EMG) Controller on Player Enjoyment of Games Designed to Teach the Use of Prosthetic Arms , 2018, The Computer Games Journal.

[47]  Nitish V. Thakor,et al.  A Mixed-Reality Training Environment for Upper Limb Prosthesis Control , 2018, 2018 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[48]  Fares Kayali,et al.  PlayBionic: Game‐Based Interventions to Encourage Patient Engagement and Performance in Prosthetic Motor Rehabilitation , 2018, PM & R : the journal of injury, function, and rehabilitation.

[49]  Robert S. Armiger,et al.  Virtual Integration Environment as an Advanced Prosthetic Limb Training Platform , 2018, Front. Neurol..

[50]  Mitchell Ruble,et al.  Mobile, Game-Based Training for Myoelectric Prosthesis Control , 2018, Front. Bioeng. Biotechnol..

[51]  Levi J Hargrove,et al.  Adapting myoelectric control in real-time using a virtual environment , 2019, Journal of NeuroEngineering and Rehabilitation.

[52]  Markus Nowak,et al.  VITA-an everyday virtual reality setup for prosthetics and upper-limb rehabilitation. , 2019, Journal of neural engineering.

[53]  Fares Kayali,et al.  MyoBeatz: Using music and rhythm to improve prosthetic control in a mobile game for health , 2019, 2019 IEEE 7th International Conference on Serious Games and Applications for Health (SeGAH).

[54]  Rajesh Krishna Balan,et al.  Design and Assessment of Myoelectric Games for Prosthesis Training of Upper Limb Amputees , 2019, 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops).

[55]  Nassir Navab,et al.  Upbeat: Augmented Reality-Guided Dancing for Prosthetic Rehabilitation of Upper Limb Amputees , 2019, Journal of healthcare engineering.

[56]  Barbara Hammer,et al.  Counteracting Electrode Shifts in Upper-Limb Prosthesis Control via Transfer Learning , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[57]  Erandi Lakshika,et al.  Prosthetic Rehabilitation Training in Virtual Reality , 2019, 2019 IEEE 7th International Conference on Serious Games and Applications for Health (SeGAH).

[58]  Kianoush Nazarpour,et al.  A HOME-BASED MYOELECTRIC TRAINING SYSTEM FOR CHILDREN , 2020 .

[59]  Egbert Otten,et al.  Performance among different types of myocontrolled tasks is not related. , 2020, Human movement science.

[60]  Cosima Prahm,et al.  Designing a Serious Game for Myoelectric Prosthesis Control , 2020, 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH).

[61]  Benoit Gosselin,et al.  Unsupervised Domain Adversarial Self-Calibration for Electromyography-Based Gesture Recognition , 2019, IEEE Access.

[62]  Vassilis Kilintzis,et al.  Serious Gaming Technology in Upper Extremity Rehabilitation: Scoping Review , 2020, JMIR serious games.

[63]  Toshio Tsuji,et al.  A New Approach for Training on EMG-based Prosthetic Hand Control , 2020, 2020 IEEE 2nd Global Conference on Life Sciences and Technologies (LifeTech).

[64]  João Almeida,et al.  The Practical Work of Ensuring the Effective Use of Serious Games in a Rehabilitation Clinic: Qualitative Study , 2020, JMIR rehabilitation and assistive technologies.

[65]  Raoul M. Bongers,et al.  Serious gaming to generate separated and consistent EMG patterns in pattern-recognition prosthesis control , 2020, Biomed. Signal Process. Control..

[66]  Nasrul Anuar Abd Razak,et al.  Comparison of Conventional and Virtual Reality Box and Blocks Tests in Upper Limb Amputees: A Case-Control Study , 2021, IEEE Access.

[67]  K. Glette,et al.  A Transferable Adaptive Domain Adversarial Neural Network for Virtual Reality Augmented EMG-Based Gesture Recognition , 2021, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[68]  Andreas W. Franzke,et al.  User training for machine learning controlled upper limb prostheses: a serious game approach , 2021, Journal of neuroengineering and rehabilitation.

[69]  John-John Cabibihan,et al.  The importance of sensory feedback to enhance embodiment during virtual training of myoelectric prostheses users , 2021, 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW).

[70]  H. Gholizadeh,et al.  Video Game–Based Rehabilitation Approach for Individuals Who Have Undergone Upper Limb Amputation: Case-Control Study , 2021, JMIR serious games.

[71]  M. Dyson,et al.  Perception of Game-Based Rehabilitation in Upper Limb Prosthetic Training: Survey of Users and Researchers , 2020, JMIR serious games.

[72]  S. Došen,et al.  Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis , 2021, Journal of neuroengineering and rehabilitation.