Serious games to improve social and emotional intelligence in children with autism

Abstract The use of serious games provides a new dimension for treating individuals with Autism spectrum disorder (ASD). Researchers have carried out a substantial amount of work in this domain with promising results in the past two decades. This study reviews the literature to design serious games to improve social behaviour in an individual with ASD. Forty serious games, developed between 2000 and 2019, were selected, and their design was evaluated based on the conceptual framework from the literature. The Connolly scale was used to evaluate the quality of the study. The analysis shows that significant work in this field has targeted children with high-functioning ASD or Asperger syndrome. Apart from general social behaviour, more focus is titled on emotion recognition/production. In the design prospect, serious games should have a rather larger sample size; it also requires clinical validation and regular follow-ups. We summarized the shortcomings in the existing serious games' design and suggested improvements that can increase serious games' effectiveness for people with ASD.

[1]  Sara Bernardini,et al.  ECHOES: An intelligent serious game for fostering social communication in children with autism , 2014, Inf. Sci..

[2]  James E. Driskell,et al.  Games, Motivation, and Learning: A Research and Practice Model , 2002 .

[3]  Zachary Wartell,et al.  Evaluation of the exertion and motivation factors of a virtual reality exercise game for children with autism , 2013, 2013 1st Workshop on Virtual and Augmented Assistive Technology (VAAT).

[4]  Valérie Gay,et al.  Building Social Awareness for Teens and Young Adults with Autism via Gamification , 2016, JCSG.

[5]  Vítor H. Carvalho,et al.  Tobias in the Zoo - A Serious Game for Children with Autism Spectrum Disorders , 2015, Int. J. Adv. Corp. Learn..

[6]  Matthew S. Goodwin,et al.  Enhancing and Accelerating the Pace of Autism Research and Treatment , 2008 .

[7]  M. Chetouani,et al.  ICT and autism care: state of the art , 2018, Current opinion in psychiatry.

[8]  Lili Cheng.  KidTalk: Online Therapy for Asperger's Syndrome , 2002 .

[9]  Jean Écalle,et al.  Serious games as new educational tools: how effective are they? A meta-analysis of recent studies , 2013, J. Comput. Assist. Learn..

[10]  Radoslava Kraleva,et al.  ChilDiBu – A Mobile Application for Bulgarian Children with Special Educational Needs , 2017 .

[11]  Matthew S. Goodwin,et al.  Technology for just-in-time in-situ learning of facial affect for persons diagnosed with an autism spectrum disorder , 2008, Assets '08.

[12]  Ernest Adams,et al.  Fundamentals of Game Design , 2006 .

[13]  Rui Prada,et al.  A Virtual Agent Toolkit for Serious Games Developers , 2018, 2018 IEEE Conference on Computational Intelligence and Games (CIG).

[14]  Daniel C. Krawczyk,et al.  Virtual Reality Social Cognition Training for children with high functioning autism , 2016, Comput. Hum. Behav..

[15]  S. Spence,et al.  Social Skills Training with Children and Young People: Theory, Evidence and Practice. , 2003, Child and adolescent mental health.

[16]  Pier J. M. Prins,et al.  Does Computerized Working Memory Training with Game Elements Enhance Motivation and Training Efficacy in Children with ADHD? , 2011, Cyberpsychology Behav. Soc. Netw..

[17]  S. Serret,et al.  Facing the challenge of teaching emotions to individuals with low- and high-functioning autism using a new Serious game: a pilot study , 2014, Molecular Autism.

[18]  Juan Pablo Hourcade,et al.  Multitouch tablet applications and activities to enhance the social skills of children with autism spectrum disorders , 2012, Personal and Ubiquitous Computing.

[19]  Shalom M. Fisch,et al.  Making educational computer games "educational" , 2005, IDC '05.

[20]  N. Bauminger,et al.  The Facilitation of Social-Emotional Understanding and Social Interaction in High-Functioning Children with Autism: Intervention Outcomes , 2002, Journal of autism and developmental disorders.

[21]  Fabio Pianesi,et al.  Collaborative Puzzle Game: a tabletop interactive game for fostering collaboration in children with Autism Spectrum Disorders (ASD) , 2009, ITS '09.

[22]  Thomas M. Connolly,et al.  The role of psychology in understanding the impact of computer games , 2011, Entertain. Comput..

[23]  Anne Marie Piper,et al.  SIDES: A Cooperative Tabletop Computer Game for Social Skills Development , 2022 .

[24]  S. Baron-Cohen,et al.  Enhancing Emotion Recognition in Children with Autism Spectrum Conditions: An Intervention Using Animated Vehicles with Real Emotional Faces , 2010, Journal of autism and developmental disorders.

[25]  J. Ducharme,et al.  Errorless Academic Compliance Training , 2012, Behavior modification.

[26]  Weiqin Chen,et al.  Multitouch Tabletop Technology for People with Autism Spectrum Disorder: A Review of the Literature , 2012, DSAI.

[27]  Matthieu Poyade,et al.  Sinbad and the Magic Cure: A Serious Game for Children with ASD and Auditory Hypersensitivity , 2017, GALA.

[28]  Emilia I. Barakova,et al.  My Drama: Story-Based Game for Understanding Emotions in Context , 2016, INTETAIN.

[29]  Jewoong Moon.  Reviews of Social Embodiment for Design of Non-Player Characters in Virtual Reality-Based Social Skill Training for Autistic Children , 2018, Multimodal Technol. Interact..

[30]  N. C. Pee,et al.  Serious Game for Autism Children: Review of Literature , 2012 .

[31]  Deborah I Thompson,et al.  Playing for real: video games and stories for health-related behavior change. , 2008, American journal of preventive medicine.

[32]  Alinda Gillott,et al.  Levels of anxiety and sources of stress in adults with autism , 2007, Journal of intellectual disabilities : JOID.

[33]  Bretagne Abirached,et al.  Understanding User Needs for Serious Games for Teaching Children with Autism Spectrum Disorders Emotions , 2012 .

[34]  Ulrike Lucke,et al.  Zirkus Empathico: Eine mobile Applikation zum Training sozioemotionaler Kompetenzen bei Kindern im Autismus-Spektrum , 2015, DeLFI.

[35]  M. Lauritsen.  Autism spectrum disorders , 2012, European Child & Adolescent Psychiatry.

[36]  K. Sofronoff,et al.  A multi-component social skills intervention for children with Asperger syndrome: the Junior Detective Training Program. , 2008, Journal of child psychology and psychiatry, and allied disciplines.

[37]  I. Shochet,et al.  Do autism spectrum disorders differ from each other and from non-spectrum disorders on emotion recognition tests? , 2001, European Child & Adolescent Psychiatry.

[38]  Mike Y. Chen,et al.  iCAN: A tablet-based pedagogical system for improving the user experience of children with autism in the learning process , 2013, 2013 1st International Conference on Orange Technologies (ICOT).

[39]  J. Tanaka,et al.  From the small screen to the big world: mobile apps for teaching real-world face recognition to children with autism , 2015 .

[40]  Jennifer Tillotson,et al.  Scentsory Design®: Scent Whisper and Fashion Fluidics , 2008 .

[41]  M. Bellani,et al.  Virtual reality in autism: state of the art , 2011, Epidemiology and Psychiatric Sciences.

[42]  Sven Bölte,et al.  Emotion recognition training in autism spectrum disorder: A systematic review of challenges related to generalizability , 2018, Developmental neurorehabilitation.

[43]  Bimlesh Wadhwa,et al.  Collaborative tablet applications to enhance language skills of children with autism spectrum disorder , 2013, APCHI.

[44]  Chek Tien Tan,et al.  Can you CopyMe?: an expression mimicking serious game , 2013, SA '13.

[45]  Joshua M. Smyth,et al.  Beyond Self-Selection in Video Game Play: An Experimental Examination of the Consequences of Massively Multiplayer Online Role-Playing Game Play , 2007, Cyberpsychology Behav. Soc. Netw..

[46]  Asadullah Shah,et al.  A systematic review of modalities in computer-based interventions (CBIs) for language comprehension and decoding skills of children with autism spectrum disorder (ASD) , 2019, Universal Access in the Information Society.

[47]  Mohamed Chetouani,et al.  GOLIAH: A Gaming Platform for Home-Based Intervention in Autism – Principles and Design , 2016, Front. Psychiatry.

[48]  Bendella Fatima,et al.  MEDIUS: A Serious Game for Autistic Children Based on Decision System , 2018, Simulation & Gaming.

[49]  N. Bauminger,et al.  Peer Interaction and Loneliness in High-Functioning Children with Autism , 2003, Journal of autism and developmental disorders.

[50]  Tiago Fernandes,et al.  LIFEisGAME: A Facial Character Animation System to Help Recognize Facial Expressions , 2011, CENTERIS.

[51]  Siti Salwah Salim,et al.  A systematic review of strategies and computer-based intervention (CBI) for reading comprehension of children with autism , 2013 .

[52]  Herre van Oostendorp,et al.  Overview of Instructional Techniques to Facilitate Learning and Motivation of Serious Games , 2017 .

[53]  Herre van Oostendorp,et al.  Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective , 2009 .

[54]  David R. Michael,et al.  Serious Games: Games That Educate, Train, and Inform , 2005 .

[55]  N. Sriram,et al.  Enhancing Social Problem Solving in Children with Autism and Normal Children Through Computer-Assisted Instruction , 2001, Journal of autism and developmental disorders.

[56]  Julie I. Cohen,et al.  Effect of action video games on the spatial distribution of visuospatial attention. , 2006, Journal of experimental psychology. Human perception and performance.

[57]  Siti Salwah Salim,et al.  A framework to design vocabulary-based serious games for children with autism spectrum disorder (ASD) , 2019, Universal Access in the Information Society.

[58]  Scott Bellini,et al.  The Development of Social Anxiety in Adolescents With Autism Spectrum Disorders , 2006 .

[59]  J. Smyth,et al.  Designing Serious Game Interventions for Individuals with Autism , 2014, Journal of Autism and Developmental Disorders.

[60]  Nyuk Hiong Voon,et al.  AutiSay: A Mobile Communication Tool for Autistic Individuals , 2014, INNS-CIIS.

[61]  Ian Pitt,et al.  Usability Evaluation of Educational Game Software for Children with Autism , 2008 .

[62]  Athanasios Drigas,et al.  Mobile Applications to Improve Emotional Intelligence in Autism - A Review , 2018, Int. J. Interact. Mob. Technol..

[63]  Erika A. Patall.  Constructing motivation through choice, interest, and interestingness , 2013 .

[64]  M. Veen,et al.  IMPROVING COLLABORATION FOR CHILDREN WITH PDD-NOS THROUGH A MULTI-TOUCH BASED SERIOUS GAME , 2009 .

[65]  Clint A. Bowers,et al.  Serious Games Usability Testing: How to Ensure Proper Usability, Playability, and Effectiveness , 2011, HCI.

[66]  Sigal Eden,et al.  NoProblem! A Collaborative Interface for Teaching Conversation Skills to Children with High Functioning Autism Spectrum Disorder , 2014 .

[67]  Begoña García Zapirain,et al.  Serious games and Health Informatics: A unified framework , 2012, 2012 17th International Conference on Computer Games (CGAMES).

[68]  Verónica Orvalho,et al.  LIFEisGAME Prototype: A Serious Game about Emotions for Children with Autism Spectrum Disorders , 2013, PsychNology J..

[69]  René Noël,et al.  Development of Software that Supports the Improvement of the Empathy in Children with Autism Spectrum Disorder , 2012, 2012 31st International Conference of the Chilean Computer Science Society.

[70]  Wilfried Admiraal,et al.  Mobile game-based learning in secondary education: engagement, motivation and learning in a mobile city game , 2009, J. Comput. Assist. Learn..

[71]  José Lozada,et al.  Tactile Emotions: A Vibrotactile Tactile Gamepad for Transmitting Emotional Messages to Children with Autism , 2012, EuroHaptics.

[72]  S. Parsons,et al.  State-of-the-art of virtual reality technologies for children on the autism spectrum , 2011 .

[73]  Pablo Moreno-Ger,et al.  Usability Testing for Serious Games: Making Informed Design Decisions with User Data , 2012, Adv. Hum. Comput. Interact..

[74]  Eleonora Bilotta,et al.  An emotional learning environment for subjects with Autism Spectrum Disorder , 2013, 2013 International Conference on Interactive Collaborative Learning (ICL).

[75]  K. Werbach,et al.  For the Win: How Game Thinking Can Revolutionize Your Business , 2012 .

[76]  Amri Yusoff,et al.  A conceptual framework for serious games and its validation , 2010 .

[77]  K. Antonopoulou,et al.  Psycho-social characteristics of children and adolescents with siblings on the autistic spectrum , 2014 .

[78]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[79]  Mateu Sbert,et al.  Serious games for health , 2013, Entertain. Comput..

[80]  M. Silver,et al.  Evaluation of a New Computer Intervention to Teach People with Autism or Asperger Syndrome to Recognize and Predict Emotions in Others , 2001, Autism : the international journal of research and practice.

[81]  S. Bölte,et al.  ‘Emotiplay’: a serious game for learning about emotions in children with autism: results of a cross-cultural evaluation , 2017, European Child & Adolescent Psychiatry.

[82]  Bretagne Abirached,et al.  A framework for designing assistive technologies for teaching children with ASDs emotions , 2012, CHI Extended Abstracts.

[83]  Niels Pinkwart,et al.  On the Adaptability and Applicability of Multi-touch User Interfaces Addressing Behavioral Interventions for Children with Autism , 2020, IETE Technical Review.

[84]  Liming Chen,et al.  JEMImE: A Serious Game to Teach Children with ASD How to Adequately Produce Facial Expressions , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[85]  Ru Zarin.  Trollskogen: a multitouch table top framework for enhancing communication amongst cognitively disabled children , 2009, ITS '09.

[86]  K. Mills,et al.  Online multiplayer games for the social interactions of children with autism spectrum disorder: a resource for inclusive education , 2018 .

[87]  Laura Malinverni,et al.  An inclusive design approach for developing video games for children with Autism Spectrum Disorder , 2017, Comput. Hum. Behav..

[88]  Michele D. Dickey.  Game Design Narrative for Learning: Appropriating Adventure Game Design Narrative Devices and Techniques for the Design of Interactive Learning Environments , 2006 .

[89]  R. Schafer,et al.  The role of autism in a visual figure-ground relationship , 1943 .

[90]  M. Wetherell,et al.  Behaviour problems of children with ASD and perceived stress in their caregivers: The moderating role of trait emotional intelligence? , 2016 .

[91]  Minhua Ma,et al.  A Review of Serious Games for Children with Autism Spectrum Disorders (ASD) , 2014, SGDA.

[92]  G. Orsmond,et al.  The daily lives of adolescents with an autism spectrum disorder , 2011, Autism : the international journal of research and practice.

[93]  Qiang Wu,et al.  CaptureMyEmotion: Helping Autistic Children Understand their Emotions Using Facial Expression Recognition and Mobile Technologies , 2013, Bled eConference.

[94]  Elisa S. Shernoff,et al.  Evidence-Based Practice: Promoting Evidence-Based Interventions in School Psychology. , 2003 .

[95]  David Cohen,et al.  Serious games to teach social interactions and emotions to individuals with autism spectrum disorders (ASD) , 2017, Comput. Educ..