Assisting Individuals with Autism and Cognitive Disorders: An Augmented Reality-Based Framework

Individuals with autism require systematic assistance while dealing with the surrounding environment and its objects. The project aim is to develop a framework that could be of substantial help to people with autism and cognitive disorders. The framework is based on common mobile devices and freely available Augmented Reality (AR) applications. The Augmented Reality used in our approach is marker-based AR that employs a camera and a visual marker to trigger and present media content on the mobile device’s screen. The developed framework allows parents and teachers to easily create educational augmented environments for children with autism and cognitive disorders by populating a real-world space with visual markers of favorite cartoonish helpers that can evoke helpful AR content and embed it in the real-world environment. The paper analyzes and discusses the use of the proposed framework from conceptual and technological points of view.

[1]  Marcello Ferro,et al.  Human-Robot Interaction in Autism: FACE, an Android-based Social Therapy , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[2]  Vladimir Geroimenko.  Augmented Reality Technology and Art: The Analysis and Visualization of Evolving Conceptual Models , 2012, 2012 16th International Conference on Information Visualisation.

[3]  Zhen Bai,et al.  Through the looking glass: Pretend play for children with autism , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[4]  Ali Pourmand,et al.  Videogame-Related Illness and Injury: A Review of the Literature and Predictions for Pokémon GO! , 2017, Games for health journal.

[5]  Jihad Mohamad Jaam,et al.  Design of Immersive Virtual Reality System to Improve Communication Skills in Individuals with Autism , 2017, Int. J. Emerg. Technol. Learn..

[6]  Fan Zhang,et al.  Toon-chat: a cartoon-masked chat system for children with autism , 2017, SIGGRAPH Posters.

[7]  Vladimir Geroimenko,et al.  Teaching Design Project in Introductory Engineering Course Using 3D Modeling and Immersive Virtual Reality , 2017 .

[8]  Entesar Hosseini,et al.  Play Therapy in Augmented Reality Children with Autism , 2016 .

[9]  M. H. Charlop-Christy,et al.  A Comparison of Video Modeling with In Vivo Modeling for Teaching Children with Autism , 2000, Journal of autism and developmental disorders.

[10]  Kevin M. Ayres,et al.  Intervention and Instruction with Video for Students with Autism: A Review of the Literature. , 2005 .

[11]  Sergio Casas,et al.  On the Development of VR and AR Learning Contents for Children on the Autism Spectrum: From Real Requirements to Virtual Scenarios , 2018 .

[12]  Murat Akcay,et al.  Advantages and challenges associated with augmented reality for education : A systematic review of the literature , 2017 .

[13]  Lenore Metrick-Chen,et al.  Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium , 2015 .

[14]  Zhen Bai,et al.  Using Augmented Reality to Elicit Pretend Play for Children with Autism , 2015, IEEE Transactions on Visualization and Computer Graphics.

[15]  J. Lutzker,et al.  Teaching Daily Living Skills to Children with Autism Through Instructional Video Modeling , 2002 .

[16]  C. Deruelle,et al.  Typical Emotion Processing for Cartoon but not for Real Faces in Children with Autistic Spectrum Disorders , 2008, Journal of autism and developmental disorders.

[17]  N. S. Lakshmiprabha,et al.  [Poster] An augmented and virtual reality system for training autistic children , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[18]  Filomena Soares,et al.  Augmented reality for cognitive and social skills improvement in children with ASD , 2016, 2016 13th International Conference on Remote Engineering and Virtual Instrumentation (REV).

[19]  Jaesung Hong,et al.  Augmented Reality in Medicine , 2016 .

[20]  Osama Halabi,et al.  Navigation Aid for Blind People Using Depth Information and Augmented Reality Technology , 2012 .

[21]  Chien-Hsu Chen,et al.  Augmented Reality Based Social Stories Training System for Promoting the Social Skills of Children with Autism , 2017 .

[22]  N. Sarkar,et al.  Design of a Virtual Reality Based Adaptive Response Technology for Children With Autism , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[23]  Uttama Lahiri,et al.  A step towards virtual reality based social communication for children with Autism , 2013, 2013 International Conference on Control, Automation, Robotics and Embedded Systems (CARE).

[24]  Uttama Lahiri,et al.  Understanding psychophysiological response to a Virtual Reality-based social communication system for children with ASD , 2011, 2011 International Conference on Virtual Rehabilitation.

[25]  Gregory D. Abowd,et al.  Exploration of videochat for children with autism , 2013, IDC.

[26]  Jihad Mohamad Jaam,et al.  Immersive Virtual Reality in Improving Communication Skills in Children with Autism , 2017, Int. J. Interact. Mob. Technol..

[27]  Lizbeth Escobedo,et al.  Mobile Augmented Reality to Support Teachers of Children with Autism , 2014, UCAmI.

[28]  Marva Angélica Mora Lumbreras,et al.  Aura: Augmented Reality in Mobile Devices for the Learning of Children With ASD – Augmented Reality in the Learning of Children With Autism , 2018 .

[29]  Rosalind W. Picard.  Future affective technology for autism and emotion communication , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.