HoloLearn: Wearable Mixed Reality for People with Neurodevelopmental Disorders (NDD)

Our research explores the potential of wearable Mixed Reality (MR) for people with Neuro-Developmental Disorders (NDD). The paper presents HoloLearn, a MR application designed in cooperation with NDD experts and implemented using HoloLens technology. The goal of HoloLearn is to help people with NDD learn how to perform simple everyday tasks in domestic environments and improve autonomy. An original feature of the system is the presence of a virtual assistant devoted to capture the user's attention and to give her/him hints during task execution in the MR environment. We performed an exploratory study involving 20 subjects with NDD to investigate the acceptability and usability of HoloLearn and its potential as a therapeutic tool. HoloLearn was well-accepted by the participants and the activities in the MR space were perceived as enjoyable, despite some usability problems associated to HoloLens interaction mechanism. More extensive and long term empirical research is needed to validate these early results, but our study suggests that HoloLearn could be adopted as a complement to more traditional interventions. Our work, and the lessons we learned, may help designers and developers of future MR applications devoted to people with NDD and to other people with similar needs.

[1]  Margot Brereton,et al.  Design insights into embedding virtual reality content into life skills training for people with intellectual disability , 2016, OZCHI.

[2]  Franca Garzotto,et al.  FaTe2: storytelling edutainment experiences in 2D and 3D collaborative spaces , 2006, IDC '06.

[3]  Michael P. Stryker,et al.  Reversing Neurodevelopmental Disorders in Adults , 2008, Neuron.

[4]  Arshya Vahabzadeh,et al.  Feasibility of an Autism-Focused Augmented Reality Smartglasses System for Social Communication and Behavioral Coaching , 2017, Front. Pediatr..

[5]  Franca Garzotto,et al.  HoloLens-based Mixed Reality Experiences for Subjects with Alzheimer's Disease , 2017, CHItaly.

[6]  Rajiv V. Dubey,et al.  Locomotion in Virtual Reality for Individuals with Autism Spectrum Disorder , 2016, SUI.

[7]  Gillian R. Hayes,et al.  SayWAT: Augmenting Face-to-Face Conversations for Adults with Autism , 2016, CHI.

[8]  Mirko Gelsomini,et al.  Wildcard: A wearable virtual reality storytelling tool for children with intellectual developmental disability , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[9]  Redwan Alqasemi,et al.  A Survey on Virtual Reality for Individuals with Autism Spectrum Disorder: Design Considerations , 2018, IEEE Transactions on Learning Technologies.

[10]  Mirko Gelsomini,et al.  Creating Social Stories as Wearable Hyper-Immersive Virtual Reality Experiences for Children with Neurodevelopmental Disorders , 2017, IDC.

[11]  D Strickland,et al.  Virtual reality for the treatment of autism. , 1997, Studies in health technology and informatics.

[12]  G. Arbanas Diagnostic and Statistical Manual of Mental Disorders (DSM-5) , 2015 .

[13]  Lucía Vera,et al.  Abstract concept and imagination teaching through Virtual Reality in people with Autism Spectrum Disorders , 2006 .

[14]  N. Newbutt,et al.  Brief Report: A Pilot Study of the Use of a Virtual Reality Headset in Autism Populations , 2016, Journal of autism and developmental disorders.

[15]  Mirko Gelsomini,et al.  Wearable Immersive Storytelling for Disabled Children , 2016, AVI.

[16]  Inmaculada Coma,et al.  A Kinect-based Augmented Reality System for Individuals with Autism Spectrum Disorders , 2018, GRAPP/IVAPP.

[17]  Sarah Parsons,et al.  Sense of presence and atypical social judgments in immersive virtual environments , 2010, Autism : the international journal of research and practice.

[18]  Gerardo Herrera,et al.  Development of symbolic play through the use of virtual reality tools in children with autistic spectrum disorders , 2008, Autism : the international journal of research and practice.

[19]  Mariano Etchart,et al.  A wearable immersive web-virtual reality approach to remote neurodevelopmental disorder therapy , 2018, AVI.

[20]  P. Mitchell,et al.  The potential of virtual reality in social skills training for people with autistic spectrum disorders. , 2002, Journal of intellectual disability research : JIDR.

[21]  Alexiei Dingli,et al.  Walking in small shoes: Investigating the power of VR on empathising with children's difficulties , 2017, 2017 23rd International Conference on Virtual System & Multimedia (VSMM).

[22]  Patrice L. Weiss,et al.  Effectiveness of virtual reality for teaching street-crossing skills to children and adolescents with autism , 2008 .

[23]  Amy D. Kyle,et al.  America's Children and the Environment: A First View of Available Measures. , 2000 .

[24]  Redwan Alqasemi,et al.  Vocational Rehabilitation of Individuals with Autism Spectrum Disorder with Virtual Reality , 2017, ACM Trans. Access. Comput..

[25]  Franca Garzotto,et al.  Empowering Interventions for Persons with Neurodevelopmental Disorders through Wearable Virtual Reality and Bio-sensors , 2018, CHI Extended Abstracts.

[26]  Mirko Gelsomini,et al.  Wearable Immersive Virtual Reality for Children with Disability: a Case Study , 2017, IDC.

[27]  Miguel Castelo-Branco,et al.  A serious game with virtual reality for travel training with Autism Spectrum Disorder , 2015, 2015 International Conference on Virtual Rehabilitation (ICVR).

[28]  Andrea Tartaro,et al.  Storytelling with a virtual peer as an intervention for children with autism , 2005, ASAC.

[29]  Franca Garzotto,et al.  Touchless Motion-Based Interaction for Therapy of Autistic Children , 2014 .