Augmented Reality for Learning of Children and Adolescents With Autism Spectrum Disorder (ASD): A Systematic Review

This paper presents a systematic review of relevant primary studies on the use of augmented reality (AR) to improve various skills of children and adolescents diagnosed with autism spectrum disorder (ASD) from years 2005 to 2018 inclusive in eight bibliographic databases. This systematic review attempts to address eleven specific research questions related to the learing skills, participants, AR technology, research design, data collection methods, settings, evaluation parameters, intervention outcomes, generalization, and maintenance. The social communication skill was the highly targeted skill, and individuals with ASD were part of all the studies. Computer, smartphone, and smartglass are more frequently used technologies. The commonly used research design was pre-test and post-test. Almost all the studies used observation as a data collection method, and classroom environment or controlled research environment were used as a setting of evaluation. Most of the evaluation parameters were human-assisted. The results of the studies show that AR benefited children with ASD in learning skills. The generalization test was conducted in one study only, but the results were not reported. The results of maintenance tests conducted in five studies during a short-term period following the withdrawal of intervention were positive. Although the effect of using AR towards the learning of individuals was positive, given the wide variety of skills targeted in the studies, and the heterogeneity of the participants, a summative conclusion regarding the effectiveness of AR for teaching or learning of skills related to ASD based on the existing literature is not possible. The review also proposes the research taxonomy for ASD. Future research addressing the effectiveness of AR among more participants, different technologies supporting AR for the intervention, generalization, and maintenance of learning skills, and the evaluation in the inslusive classroom environment and other settings is warranted.

[1]  Fan Zhang,et al.  Toon-chat: a cartoon-masked chat system for children with autism , 2017, SIGGRAPH Posters.

[2]  L. Billeci,et al.  Autism and social robotics: A systematic review , 2016, Autism research : official journal of the International Society for Autism Research.

[3]  Taryadi,et al.  The improvement of autism spectrum disorders on children communication ability with PECS method Multimedia Augmented Reality-Based , 2018 .

[4]  Weiqin Chen,et al.  Multitouch Tabletop Technology for People with Autism Spectrum Disorder: A Review of the Literature , 2012, DSAI.

[5]  Stelios Xinogalos,et al.  Studying the effects of computer serious games on people with intellectual disabilities or autism spectrum disorder: A systematic literature review , 2018, J. Comput. Assist. Learn..

[6]  Nicola Yuill,et al.  An augmented toy and social interaction in children with autism , 2012, Int. J. Arts Technol..

[7]  Chien-Hsu Chen,et al.  Augmented Reality Based Social Stories Training System for Promoting the Social Skills of Children with Autism , 2017 .

[8]  Jessica Clarkson DEVELOPMENT OF AN IOS APP USING SITUATED LEARNING, COMMUNITIES OF PRACTICE, AND AUGMENTED REALITY FOR AUTISM SPECTRUM DISORDER , 2014 .

[9]  Marva Angélica Mora Lumbreras,et al.  Aura: Augmented Reality in Mobile Devices for the Learning of Children With ASD – Augmented Reality in the Learning of Children With Autism , 2018 .

[10]  Arshya Vahabzadeh,et al.  Improvement of Attention-Deficit/Hyperactivity Disorder Symptoms in School-Aged Children, Adolescents, and Young Adults With Autism via a Digital Smartglasses-Based Socioemotional Coaching Aid: Short-Term, Uncontrolled Pilot Study , 2018, JMIR mental health.

[11]  Lakhmi C. Jain,et al.  Future Trends of Virtual, Augmented Reality, and Games for Health , 2014 .

[12]  William M. K. Trochim,et al.  Research methods knowledge base , 2001 .

[13]  Helen Pain,et al.  Facilitating Pretend Play in Autistic Children: Results from an Augmented Reality App Evaluation , 2018, ASSETS.

[14]  Iulian Radu,et al.  Augmented reality in education: a meta-review and cross-media analysis , 2014, Personal and Ubiquitous Computing.

[15]  Morten Fjeld,et al.  Exploring Diminished Reality (DR) Spaces to Augment the Attention of Individuals with Autism , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality Workshops.

[16]  Chien-Hsu Chen,et al.  Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. , 2015, Research in developmental disabilities.

[17]  Constantine Stephanidis,et al.  Let's Cook: An Augmented Reality System Towards Developing Cooking Skills for Children with Cognitive Impairments , 2017, GOODTECHS.

[18]  Arshya Vahabzadeh,et al.  Second Version of Google Glass as a Wearable Socio-Affective Aid: Positive School Desirability, High Usability, and Theoretical Framework in a Sample of Children with Autism , 2018, JMIR human factors.

[19]  E. Goldson Educating Children with Autism , 2004 .

[20]  Thies Pfeiffer,et al.  Augmented reality action assistance and learning for cognitively impaired people: a systematic literature review , 2019, PETRA.

[21]  S. Fletcher‐Watson A Targeted Review of Computer-Assisted Learning for People with Autism Spectrum Disorder: Towards a Consistent Methodology , 2013, Review Journal of Autism and Developmental Disorders.

[22]  Minhua Ma,et al.  A Review of Serious Games for Children with Autism Spectrum Disorders (ASD) , 2014, SGDA.

[23]  Lizbeth Escobedo,et al.  Using Augmented Reality to Help Children with Autism Stay Focused , 2014, IEEE Pervasive Computing.

[24]  Christopher Frauenberger,et al.  Blending Methods: Developing Participatory Design Sessions for Autistic Children , 2017, IDC.

[25]  António Ramires Fernandes,et al.  STAR: Speech Therapy with Augmented Reality for Children with Autism Spectrum Disorders , 2014, ICEIS.

[26]  Mohammad Efendi,et al.  The Book of Pop Up Augmented Reality to Increase Focus and Object Recognition Capabilities for Children with Autism , 2018 .

[27]  B. Garcia-Zapirain,et al.  Technologies as Support Tools for Persons with Autistic Spectrum Disorder: A Systematic Review , 2014, International journal of environmental research and public health.

[28]  Diana Hammond,et al.  Descriptive Analysis of Single Subject Research Designs: 1983-2007. , 2010 .

[29]  Josep Maria Sanahuja Gavaldà,et al.  Improving the Process of Inclusive Education in Children with ASD in Mainstream Schools , 2012 .

[30]  Lizbeth Escobedo,et al.  A Smart Environment for Children with Autism , 2015, IEEE Pervasive Computing.

[31]  M F Syahputra,et al.  Augmented reality social story for autism spectrum disorder , 2018 .

[32]  Zhen Bai,et al.  Making Pretense Visible and Graspable: An augmented reality approach to promote pretend play , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[33]  Esat Alpay,et al.  A Taxonomy for Virtual and Augmented Reality in Education , 2019, ArXiv.

[34]  Barbara Kitchenham,et al.  Procedures for Performing Systematic Reviews , 2004 .

[35]  Franca Garzotto,et al.  HoloLearn: Learning through Mixed Reality for People with Cognitive Disability , 2018, 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR).

[36]  Siti Salwah Salim,et al.  A systematic review of strategies and computer-based intervention (CBI) for reading comprehension of children with autism , 2013 .

[37]  Carlos Delgado Kloos,et al.  Augmented reality for STEM learning: A systematic review , 2018, Comput. Educ..

[38]  Victoria Knight,et al.  A Review of Technology-Based Interventions to Teach Academic Skills to Students with Autism Spectrum Disorder , 2013, Journal of Autism and Developmental Disorders.

[39]  P. Richard,et al.  Augmented Reality for Rehabilitation of Cognitive Disabled Children: A Preliminary Study , 2007, 2007 Virtual Rehabilitation.

[40]  Arshya Vahabzadeh,et al.  Social Communication Coaching Smartglasses: Well Tolerated in a Diverse Sample of Children and Adults With Autism , 2017, JMIR mHealth and uHealth.

[41]  C. Frauenberger,et al.  Designing technology for children with special needs: bridging perspectives through participatory design , 2011 .

[42]  M. O'Reilly,et al.  Use of Computer-Based Interventions to Teach Communication Skills to Children with Autism Spectrum Disorders: A Systematic Review , 2011 .

[43]  Adel Al-Jumaily,et al.  Augmented Reality Game Therapy For Children With Autism Spectrum Disorder , 2017 .

[44]  Fan Zhang,et al.  An EEG-based Adaptive Training System for ASD Children , 2017, UIST.

[45]  Chin-Chung Tsai,et al.  Affordances of Augmented Reality in Science Learning: Suggestions for Future Research , 2012, Journal of Science Education and Technology.

[46]  Ferdian Ronilaya,et al.  The evaluation on a mobile augmented reality application as therapy media , 2018 .

[47]  Don D. McMahon,et al.  Effects of Digital Navigation Aids on Adults With Intellectual Disabilities , 2015 .

[48]  Sen-Ching S. Cheung,et al.  LittleHelper: An augmented reality glass application to assist individuals with autism in job interview , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[49]  M. Schijven,et al.  Systematic review on the effectiveness of augmented reality applications in medical training , 2016, Surgical Endoscopy.

[50]  N. C. Pee,et al.  Serious Game for Autism Children: Review of Literature , 2012 .

[51]  Didem Alsancak Sırakaya,et al.  Trends in Educational Augmented Reality Studies: A Systematic Review , 2018 .

[52]  Esteban MENÉNDEZ,et al.  Augmented Reality as Visual Communication for People with ASD , 2018 .

[53]  J ChungPeter,et al.  Social Behaviors and Active Videogame Play in Children with Autism Spectrum Disorder , 2015 .

[54]  Maizam Alias,et al.  A Review on Using Assistive Technology to Enhance Social Skills Competence Among Children with Autism Spectrum Disorder (ASD) , 2018, Advanced Science Letters.

[55]  Zhen Bai,et al.  Using Augmented Reality to Elicit Pretend Play for Children with Autism , 2015, IEEE Transactions on Visualization and Computer Graphics.

[56]  M. O'Reilly,et al.  Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: A systematic review , 2012, Developmental neurorehabilitation.

[57]  Chien-Hsu Chen,et al.  Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions , 2016, Comput. Hum. Behav..

[58]  Jose A. Piedra-Fernández,et al.  KiNEEt: application for learning and rehabilitation in special educational needs , 2018, Multimedia Tools and Applications.

[59]  António Ramires Fernandes,et al.  Assisting Speech Therapy for Autism Spectrum Disorders with an Augmented Reality Application , 2014, ICEIS.

[60]  Hilary Johnson,et al.  Developing IDEAS: supporting children with autism within a participatory design team , 2012, CHI.

[61]  Filomena Soares,et al.  An Augmented Reality GameBook for Children with Autism Spectrum Disorders , 2015 .

[62]  Tamila H. Kolomoiets,et al.  Using the Augmented Reality to Teach of Global Reading of Preschoolers with Autism Spectrum Disorders , 2018, AREdu.

[63]  Murat Akcay,et al.  Advantages and challenges associated with augmented reality for education : A systematic review of the literature , 2017 .

[64]  H. Almeida,et al.  Using Augmented Reality in Patients with Autism: A Systematic Review , 2019, VipIMAGE 2019.

[65]  Filomena Soares,et al.  Augmented reality for cognitive and social skills improvement in children with ASD , 2016, 2016 13th International Conference on Remote Engineering and Virtual Instrumentation (REV).

[66]  Minoru Asada,et al.  Autism simulator employing augmented reality: A prototype , 2014, 4th International Conference on Development and Learning and on Epigenetic Robotics.

[67]  Asadullah Shah,et al.  Use of augmented reality for social communication skills in children and adolescents with autism spectrum disorder (ASD): A systematic review , 2019, 2019 IEEE 6th International Conference on Engineering Technologies and Applied Sciences (ICETAS).

[68]  Inmaculada Coma,et al.  A Kinect-based Augmented Reality System for Individuals with Autism Spectrum Disorders , 2018, GRAPP/IVAPP.

[69]  Tonya N. Davis,et al.  Using iPods(®) and iPads(®) in teaching programs for individuals with developmental disabilities: a systematic review. , 2013, Research in developmental disabilities.

[70]  Arshya Vahabzadeh,et al.  Safety and Lack of Negative Effects of Wearable Augmented-Reality Social Communication Aid for Children and Adults with Autism , 2017, bioRxiv.

[71]  Gonzalo Lorenzo,et al.  Preliminary study of augmented reality as an instrument for improvement of social skills in children with autism spectrum disorder , 2018, Education and Information Technologies.

[72]  Lauren M. Schmitt,et al.  The Clinical Use of Robots for Individuals with Autism Spectrum Disorders: A Critical Review. , 2012, Research in autism spectrum disorders.

[73]  Don D. McMahon,et al.  Evaluating Augmented Reality to Complete a Chain Task for Elementary Students With Autism , 2016 .

[74]  Wen Yi,et al.  A critical review of virtual and augmented reality (VR/AR) applications in construction safety , 2018 .

[75]  Mohammed Aldosari,et al.  Prevalence and correlates of autism spectrum disorder in Qatar: a national study , 2019, Journal of child psychology and psychiatry, and allied disciplines.

[76]  Zhen Bai,et al.  Through the looking glass: Pretend play for children with autism , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[77]  Fan Zhang,et al.  KinToon: A Kinect Facial Projector for Communication Enhancement for ASD Children , 2017, UIST.

[78]  A. Jackulin Mahariba,et al.  Face Morphing and Substitution for Aid of Autistic Children using Augmented Reality , 2016 .

[79]  Olivier Nannipieri,et al.  New Augmented Reality Taxonomy: Technologies and Features of Augmented Environment , 2011, Handbook of Augmented Reality.

[80]  Siti Salwah Salim,et al.  Serious Game for Children with Autism to Learn Vocabulary: An Experimental Evaluation , 2019, Int. J. Hum. Comput. Interact..

[81]  Rincon M. Nubia,et al.  Development of a mobile application in augmented reality to improve the communication field of autistic children at a Neurorehabilitar Clinic , 2015, 2015 Workshop on Engineering Applications - International Congress on Engineering (WEA).

[82]  Hendra Pradibta,et al.  Designing AR daily prayers for children with ASD , 2017, 2017 International Seminar on Application for Technology of Information and Communication (iSemantic).

[83]  D. Massaro,et al.  Development and Evaluation of a Computer-Animated Tutor for Vocabulary and Language Learning in Children with Autism , 2003, Journal of autism and developmental disorders.

[84]  Constantine Stephanidis,et al.  Universal Access in Human-Computer Interaction. User and Context Diversity , 2013, Lecture Notes in Computer Science.

[85]  Maria Claudia Buzzi,et al.  ICT to Support Dental Care of Children with Autism: An Exploratory Study , 2018, HCI.

[86]  José-Antonio Lozano-Quilis,et al.  Effectiveness of Virtual Reality for Children and Adolescents with Autism Spectrum Disorder: An Evidence-Based Systematic Review , 2018, Sensors.

[87]  Asunción Lledó,et al.  The application of immersive virtual reality for students with ASD: A review between 1990–2017 , 2018, Education and Information Technologies.

[88]  Jorge Bacca,et al.  Augmented Reality Trends in Education: A Systematic Review of Research and Applications , 2014, J. Educ. Technol. Soc..

[89]  F. Benzarti,et al.  Application of Virtual Reality and Augmented Reality in Psychiatry and Neuropsychology, in Particular in the Case of Autistic Spectrum Disorder (ASD) , 2018, Neurophysiology.

[90]  Asadullah Shah,et al.  A systematic review of modalities in computer-based interventions (CBIs) for language comprehension and decoding skills of children with autism spectrum disorder (ASD) , 2019, Universal Access in the Information Society.

[91]  Arshya Vahabzadeh,et al.  Feasibility of an Autism-Focused Augmented Reality Smartglasses System for Social Communication and Behavioral Coaching , 2017, Front. Pediatr..

[92]  Patrick C. Shih,et al.  Puzzle Walk: A Gamified Mobile App to Increase Physical Activity in Adults with Autism Spectrum Disorder , 2018, 2018 10th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games).

[93]  N. Sarat Chandra Babu,et al.  Assistive Learning for Children with Autism Using Augmented Reality , 2013, 2013 IEEE Fifth International Conference on Technology for Education (t4e 2013).

[94]  M. O'Reilly,et al.  Use of Computer-Based Interventions to Improve Literacy Skills in Students with Autism Spectrum Disorders: A Systematic Review , 2011 .

[95]  E. Walker,et al.  Diagnostic and Statistical Manual of Mental Disorders , 2013 .

[96]  Douglas L Vanderbilt,et al.  Social Behaviors and Active Videogame Play in Children with Autism Spectrum Disorder. , 2015, Games for health journal.

[97]  Fatih Saltan,et al.  The Use of Augmented Reality in Formal Education: A Scoping Review. , 2017 .

[98]  Matthew S. Goodwin,et al.  Lessons from participatory design with adolescents on the autism spectrum , 2009, CHI Extended Abstracts.

[99]  A. Irvine,et al.  Inclusive education and best practice for children with autism spectrum disorder: an integrated approach , 2009 .

[100]  Don D. McMahon,et al.  Augmented Reality as a Navigation Tool to Employment Opportunities for Postsecondary Education Students With Intellectual Disabilities and Autism , 2015 .

[101]  Don D. McMahon,et al.  Augmented Reality for Teaching Science Vocabulary to Postsecondary Education Students With Intellectual Disabilities and Autism , 2016 .

[102]  Chien-Hsu Chen,et al.  Augmented Reality Plus Concept Map Technique to Teach Children with ASD to Use Social Cues When Meeting and Greeting , 2018, The Asia-Pacific Education Researcher.