Serious games for health and medicine. A cardiopulmonary resuscitation (CPR) case study

Serious games are video games designed to provide specific expertise, knowledge and skills to the player, going beyond pure entertainment. In this thesis we present a classification of serious games for health and medicine that have been proposed in the last decade. We propose LISSA a serious game designed to teach the cardiopulmonar resuscitation protocol (RCP). In the context of LISSA, we study the visual realism of 3D serious games and their elements, evaluating photorealistic and non-photorealistic effects and camera position in first and third person. In addition, we study the physical realism to improve the interaction between game and player. We propose a new technique based on Kinect able to reproduce the key factors of the CPR protocol. Finally, we evaluate LISSA in a real scenario

[1]  B. Bloom.  Taxonomy of educational objectives , 1956 .

[2]  E. Salas,et al.  Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. , 1993 .

[3]  James A. Ferwerda,et al.  Three varieties of realism in computer graphics , 2003, IS&T/SPIE Electronic Imaging.

[4]  J. M. Cash.  ABC of resuscitation , 1987 .

[5]  Carl Gutwin,et al.  The effect of temporal adaptation granularity and game genre on the time-balancing abilities of adaptive time-varying minigames , 2014, Entertain. Comput..

[6]  Federico Semeraro,et al.  iCPR: a new application of high-quality cardiopulmonary resuscitation training. , 2011, Resuscitation.

[7]  Begoña García Zapirain,et al.  Can game-based therapies be trusted? Is game-based education effective? A systematic review of the Serious Games for health and education , 2011, 2011 16th International Conference on Computer Games (CGAMES).

[8]  Katerina Mania,et al.  The effect of visual and interaction fidelity on spatial cognition in immersive virtual environments , 2006, IEEE Transactions on Visualization and Computer Graphics.

[9]  Mehrdad Hosseini Zadeh,et al.  Advances in Haptics , 2010 .

[10]  M. Vazquez-Vazquez,et al.  Hand hygiene training through a serious game: New ways of improving Safe Practices , 2011, 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

[11]  Abdullah Bade,et al.  Non-photorealistic outdoor scene rendering: techniques and application , 2004 .

[12]  James F. Peters,et al.  Store-and-Feedforward Adaptive Gaming System for Hand-Finger Motion Tracking in Telerehabilitation , 2011, IEEE Transactions on Information Technology in Biomedicine.

[13]  Clint Bowers,et al.  Computer simulation games as an adjunct for treatment in male veterans with alcohol use disorder. , 2013, Journal of substance abuse treatment.

[14]  Marilyn A Roubidoux,et al.  Development and evaluation of an interactive Web-based breast imaging game for medical students. , 2002, Academic radiology.

[15]  W. B. Kouwenhoven,et al.  Closed-chest cardiac massage. , 1960, JAMA.

[16]  Didier Stricker,et al.  Advanced tracking through efficient image processing and visual-inertial sensor fusion , 2008, 2008 IEEE Virtual Reality Conference.

[17]  Shaun J. Canavan,et al.  Art Critic: Multisignal Vision and Speech Interaction System in a Gaming Context , 2013, IEEE Transactions on Cybernetics.

[18]  Ana Paiva,et al.  Virtual Worlds for Serious Applications ( VS-GAMES ' 12 ) Serious Games for Personal and Social Learning & Ethics : Status and Trends , 2012 .

[19]  Jesus Reseco Gago,et al.  Nutritional Serious-Games platform , 2010, eChallenges e-2010 Conference.

[20]  Zhen Jiao,et al.  Tutoring Strategy Study on Game-Based Experiential Learning in Vocational School , 2009, 2009 First International Workshop on Education Technology and Computer Science.

[21]  Arto Laikari,et al.  Exergaming - Gaming for health: A bridge between real world and virtual communities , 2009, 2009 IEEE 13th International Symposium on Consumer Electronics.

[22]  Leandro A. Diehl,et al.  A serious game prototype for education of medical doctors and students on insulin management for treatment of diabetes mellitus , 2011, 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

[23]  Robert A. Cross.  Interactive realism for visualization using ray tracing , 1995, Proceedings Visualization '95.

[24]  Xiao Cheng-yong,et al.  Constructing 3D Game Engine Based on XNA , 2010 .

[25]  Pheng-Ann Heng,et al.  A Serious Game for Learning Ultrasound-Guided Needle Placement Skills , 2012, IEEE Transactions on Information Technology in Biomedicine.

[26]  Cheryl A.M. Anderson.  Simulation Game Playing – A Nursing Instructional Strategy , 2008 .

[27]  Fotis Liarokapis,et al.  A Pervasive Augmented Reality Serious Game , 2009, 2009 Conference in Games and Virtual Worlds for Serious Applications.

[28]  Steve Wright.  Chapter 6 – Rotoscoping , 2008 .

[29]  Tiffany Barnes,et al.  Astrojumper: Designing a virtual reality exergame to motivate children with autism to exercise , 2010, 2010 IEEE Virtual Reality Conference (VR).

[30]  Dongho Kim,et al.  A Sensory Gate-Ball Game for the Aged People and its User Interface Design , 2009, 2009 Conference in Games and Virtual Worlds for Serious Applications.

[31]  Robert Pepperell,et al.  The perception of art and the science of perception , 2012, Electronic Imaging.

[32]  Jannicke Baalsrud Hauge,et al.  State of the art of serious games for business and industry , 2011, 2011 17th International Conference on Concurrent Enterprising.

[33]  Peng Yun-long.  Principles of Teaching in the New Era , 2008 .

[34]  Z. L. Budrikis,et al.  Visual fidelity criterion and modeling , 1972 .

[35]  G J Kost,et al.  Preventing medical errors in point-of-care testing: security, validation, safeguards, and connectivity.. , 2001, Archives of pathology & laboratory medicine.

[36]  Thad Starner,et al.  Dancing in the Streets: The design and evaluation of a wearable health game , 2010, International Symposium on Wearable Computers (ISWC) 2010.

[37]  Gavin D. Perkins,et al.  Does use of iResus© application on a smart phone improve the performance of an advanced life support provider in a simulated emergency? , 2010 .

[38]  Misha Pavel,et al.  Divided attention in computer game play: Analysis utilizing unobtrusive health monitoring , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[39]  M Kalantzis,et al.  The conditions of learning , 2005 .

[40]  K. Nosu,et al.  Real Time Emotion-Diagnosis of Video Game Players from their Facial Expressions and its Applications to Voice Feed-Backing to Game Players , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[41]  Po-Hsun Cheng,et al.  Augmented reality serious game framework for rehabilitation with personal health records , 2011, 2011 IEEE 13th International Conference on e-Health Networking, Applications and Services.

[42]  Jungkuk Kim,et al.  A design of the u-health monitoring system using a Nintendo DS game machine , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[43]  Mahen Jayawardena,et al.  User Perception of the Physical & Behavioral Realism of a Maritime Virtual Reality Environment , 2012, 2012 UKSim 14th International Conference on Computer Modelling and Simulation.

[44]  J. D. Janssen,et al.  A triaxial accelerometer and portable data processing unit for the assessment of daily physical activity , 1997, IEEE Transactions on Biomedical Engineering.

[45]  David Murphy,et al.  JDoc: A Serious Game for Medical Learning , 2008, First International Conference on Advances in Computer-Human Interaction.

[46]  Luca Chittaro,et al.  Assessing Nurses' Acceptance of a Serious Game for Emergency Medical Services , 2010, 2010 Second International Conference on Games and Virtual Worlds for Serious Applications.

[47]  N. Cagiltay,et al.  A serious game for speech disorder children therapy , 2012, 2012 7th International Symposium on Health Informatics and Bioinformatics.

[48]  Suzana Ahmad,et al.  Measuring usability of educational computer games based on the user success rate , 2011, 2011 International Symposium on Humanities, Science and Engineering Research.

[49]  Ombretta Gaggi,et al.  PlayWithEyes: A new way to test children eyes , 2011, 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

[50]  V. Lakshmi Narasimhan,et al.  Design of an introductory medical gaming environment for diagnosis and management of Parkinson's disease , 2010, Trendz in Information Sciences & Computing(TISC2010).

[51]  Bill Kapralos,et al.  Serious games for knee replacement surgery procedure education and training , 2010 .

[52]  J. Huizinga.  Homo Ludens: A Study of the Play-Element in Culture , 1938 .

[53]  Darrell Gibson,et al.  Towards the Development of an Interactive 3D Coach Training Serious Game , 2009, 2009 Conference in Games and Virtual Worlds for Serious Applications.

[54]  Sergi Grau,et al.  PREVIRNEC: A cognitive telerehabilitation system based on Virtual Environments , 2009, 2009 Virtual Rehabilitation International Conference.

[55]  George Papagiannakis,et al.  Interactive Scenario Immersion: Health Emergency Decision Training in JUST Project , 2002 .

[56]  Nuno F. Rodrigues,et al.  A laparoscopic surgery training interface , 2011, 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

[57]  Maurice Hendrix,et al.  Blended Game-Based Learning Environments: Extending a Serious Game into a Learning Content Management System , 2011, 2011 Third International Conference on Intelligent Networking and Collaborative Systems.

[58]  Abdenour Bouzouane,et al.  Serious games in cognitive training for Alzheimer's patients , 2011, 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

[59]  Ton A. M. Spil,et al.  Change your lifestyle or your game is over: The design of a serious game for diabetes , 2011, International Conference on Serious Games and Applications for Health.

[60]  Kalle Jegers.  Investigating the Applicability of Usability and Playability Heuristics for Evaluation of Pervasive Games , 2008, 2008 Third International Conference on Internet and Web Applications and Services.

[61]  Tiia Ruutmann,et al.  Teaching Strategies for Direct and Indirect Instruction in Teaching Engineering , 2011 .

[62]  Suzana Ahmad,et al.  Usability testing for educational computer game using observation method , 2010, 2010 International Conference on Information Retrieval & Knowledge Management (CAMP).

[63]  Huang Ling Yi,et al.  Designing Serious Games to Enhance Political Efficacy and Critical Thinking Disposition for College Students: The Case of Taiwan , 2011, 2011 Third International Conference on Games and Virtual Worlds for Serious Applications.

[64]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[65]  Qiang Zhang,et al.  Component-based hierarchical state machine — A reusable and flexible game AI technology , 2011, 2011 6th IEEE Joint International Information Technology and Artificial Intelligence Conference.

[66]  Cheng-Wen Lee,et al.  Influential Factors of Player's Loyalty toward Online Games for Achieving Commercial Success , 2010 .

[67]  Alf Inge Wang,et al.  An Evaluation of a Mobile Game Concept for Lectures , 2008, 2008 21st Conference on Software Engineering Education and Training.

[68]  Paul Fergus,et al.  A framework for physical health improvement using Wireless Sensor Networks and gaming , 2009, 2009 3rd International Conference on Pervasive Computing Technologies for Healthcare.

[69]  Heinrich H. Bülthoff,et al.  Render me real? , 2012, ACM Trans. Graph..

[70]  Fred Hapgood.  The quest for oil , 1989 .

[71]  Huosheng Hu,et al.  Human motion tracking for rehabilitation - A survey , 2008, Biomed. Signal Process. Control..

[72]  David R. Michael,et al.  Serious Games: Games That Educate, Train, and Inform , 2005 .

[73]  Peter Jensen,et al.  At this end of the Oregon trail: The eugene AUGUR: 1969–1974 , 1990 .

[74]  Claudio E. Palazzi,et al.  DroidGlove: An android-based application for wrist rehabilitation , 2009, 2009 International Conference on Ultra Modern Telecommunications & Workshops.

[75]  Jonathan Barr,et al.  Nursing Training: 3D Game with Learning Objectives , 2008, First International Conference on Advances in Computer-Human Interaction.

[76]  E.C.P. van Loon,et al.  Serious gaming to improve bimanual coordination in children with spastic cerebral palsy , 2011, 2011 International Conference on Virtual Rehabilitation.

[77]  Rafael Bidarra,et al.  In Press: Ieee Transactions on Computational Intelligence and Ai in Games Adaptivity Challenges in Games and Simulations: a Survey , 2022 .

[78]  Marina Papastergiou,et al.  Exploring the potential of computer and video games for health and physical education: A literature review , 2009, Comput. Educ..

[79]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[80]  Stepán Obdrzálek,et al.  Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[81]  A. Maslow.  Motivation and Personality , 1954 .

[82]  Darryl Charles,et al.  Serious Games for Upper Limb Rehabilitation Following Stroke , 2009, 2009 Conference in Games and Virtual Worlds for Serious Applications.

[83]  Qiang Wang,et al.  EEG-Based "Serious" Games Design for Medical Applications , 2010, 2010 International Conference on Cyberworlds.

[84]  Elmedin Selmanovic,et al.  Complete Motion Control of a Serious Game against Obesity in Children , 2011, 2011 Third International Conference on Games and Virtual Worlds for Serious Applications.

[85]  Nicole C. Krämer,et al.  Serious games for health: An empirical study of the game "Balance" for teenagers with diabetes mellitus , 2011, Entertain. Comput..

[86]  Jeffrey Wood,et al.  Feasibility of congestive heart failure telemanagement using a wii-based telecare platform , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[87]  Pamela M. Kato,et al.  Video Games in Health Care: Closing the Gap , 2010 .

[88]  Gabriel Zachmann,et al.  'Visual-fidelity' dataglove calibration , 2004 .

[89]  Jeff Housman,et al.  The Alameda County Study: A Systematic, Chronological Review , 2005 .

[90]  Yao-Jen Chang,et al.  A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. , 2011, Research in developmental disabilities.

[91]  Jaana Leikas,et al.  Human-Centred Design and Exercise Games , 2009 .

[92]  Vasudevan Janarthanan.  Serious Video Games: Games for Education and Health , 2012, 2012 Ninth International Conference on Information Technology - New Generations.

[93]  Begoña García Zapirain,et al.  Serious Game based on first aid education for individuals with Autism Spectrum Disorder (ASD) using android mobile devices , 2011, 2011 16th International Conference on Computer Games (CGAMES).

[94]  Miriam Vollenbroek-Hutten,et al.  Chronic pain rehabilitation with a serious game using multimodal input , 2011, 2011 International Conference on Virtual Rehabilitation.

[95]  John Lansdown,et al.  Expressive rendering: a review of nonphotorealistic techniques , 1995, IEEE Computer Graphics and Applications.

[96]  Garrett M. Clayton,et al.  Integrating the Microsoft Kinect With Simulink: Real-Time Object Tracking Example , 2014, IEEE/ASME Transactions on Mechatronics.

[97]  Pedro Miguel Moreira,et al.  Serious games for rehabilitation: A survey and a classification towards a taxonomy , 2010, 5th Iberian Conference on Information Systems and Technologies.

[98]  Yin Li,et al.  An Accelerated Human Motion Tracking System Based on Voxel Reconstruction under Complex Environments , 2009, ACCV.

[99]  Clint A. Bowers,et al.  Serious Games Usability Testing: How to Ensure Proper Usability, Playability, and Effectiveness , 2011, HCI.

[100]  Robert J. Stone,et al.  Evolution of aerospace simulation: From immersive Virtual Reality to serious games , 2011, Proceedings of 5th International Conference on Recent Advances in Space Technologies - RAST2011.

[101]  Gonzalo Alarcón,et al.  Introduction to Epilepsy: Epidemiology of epilepsy , 2012 .

[102]  Sergi Grau,et al.  Animation of 3D Avatars for Rehabilitation of the Upper Limbs , 2011, 2011 Third International Conference on Games and Virtual Worlds for Serious Applications.

[103]  Henry Been-Lirn Duh,et al.  Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[104]  R. Tate,et al.  HopeLab's Approach to Re-Mission , 2009 .

[105]  Sergi Grau,et al.  Design of 3D Virtual Neuropsychological Rehabilitation Activities , 2010, 2010 Second International Conference on Games and Virtual Worlds for Serious Applications.

[106]  Peter H. Veltink,et al.  Ambulatory Position and Orientation Tracking Fusing Magnetic and Inertial Sensing , 2007, IEEE Transactions on Biomedical Engineering.

[107]  Rosa María Baños,et al.  Treating cockroach phobia using a serious game on a mobile phone and augmented reality exposure: A single case study , 2011, Comput. Hum. Behav..

[108]  Begoña García Zapirain,et al.  A serious game to improve human relationships in patients with neuro-psychological disorders , 2010, 2010 2nd International IEEE Consumer Electronics Society's Games Innovations Conference.

[109]  Antonio Frisoli,et al.  Motion detection technology as a tool for cardiopulmonary resuscitation (CPR) quality training: a randomised crossover mannequin pilot study. , 2013, Resuscitation.

[110]  Hyun-Jung Park,et al.  A Bayesian network approach to examining key success factors of mobile games , 2013 .

[111]  L. Gostin,et al.  A public health approach to reducing error: medical malpractice as a barrier. , 2000, JAMA.

[112]  Linda Denehy,et al.  Validity of the Microsoft Kinect for assessment of postural control. , 2012, Gait & posture.

[113]  Theresa-Marie Rhyne.  Computer games and scientific visualization , 2002, CACM.

[114]  Kup-Sze Choi,et al.  Learning Blood Management in Orthopedic Surgery through Gameplay , 2010, IEEE Computer Graphics and Applications.

[115]  Michael A. Shepherd,et al.  Extending the Use of Games in Health Care , 2006, Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06).