vrSocial: Toward Immersive Therapeutic VR Systems for Children with Autism

Social communication frequently includes nuanced nonverbal communication cues, including eye contact, gestures, facial expressions, body language, and tone of voice. This type of communication is central to face-to-face interaction, but can be challenging for children and adults with autism. Innovative technologies can provide support by augmenting human-delivered cuing and automated prompting. Specifically, immersive virtual reality (VR) offers an option to generalize social skill interventions by concretizing nonverbal information in real-time social interactions. In this work, we explore the design and evaluation of three nonverbal communication applications in immersive VR. The results of this work indicate that delivering real-time visualizations of proximity, speaker volume, and duration of one's speech is feasible in immersive VR and effective for real-time support for proximity regulation for children with autism. We conclude with design considerations for therapeutic VR systems.

[1]  Charlotte Brownlow,et al.  Mapping the social geographies of autism – online and off-line narratives of neuro-shared and separate spaces , 2013 .

[2]  Peter Washington,et al.  Superpower glass: delivering unobtrusive real-time social cues in wearable systems , 2016, UbiComp Adjunct.

[3]  Dan Saffer,et al.  Microinteractions: Designing with Details , 2013 .

[4]  Robert E. Kraut,et al.  Social use of computer-mediated communication by adults on the autism spectrum , 2010, CSCW '10.

[5]  Xueni Pan,et al.  Reduced Mimicry to Virtual Reality Avatars in Autism Spectrum Disorder , 2016, Journal of autism and developmental disorders.

[6]  Sri Hastuti Kurniawan,et al.  Review of Interaction design , 2003 .

[7]  B. Myles,et al.  The Hidden Curriculum: Practical Solutions for Understanding Unstated Rules in Social Situations , 2004 .

[8]  Gillian R. Hayes,et al.  SayWAT: Augmenting Face-to-Face Conversations for Adults with Autism , 2016, CHI.

[9]  Terry Winograd,et al.  A practical approach to real-time neutral feature subtraction for facial expression recognition , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Kelly Caine,et al.  Understanding Your Users: A Practical Guide to User Research Methods , 2015 .

[11]  Karrie Karahalios,et al.  Seeing More: Visualizing Audio Cues , 2007, INTERACT.

[12]  Daniel C. Krawczyk,et al.  Virtual Reality Social Cognition Training for children with high functioning autism , 2016, Comput. Hum. Behav..

[13]  A. Thiselton.  The Two Horizons: New Testament Hermeneutics and Philosophical Description with Special Reference to Heidegger, Bultmann, Gadamer and Wittgenstein , 1980 .

[14]  S. Schneiders.  The Two Horizons: New Testament Hermeneutics and Philosophical Description with Special Reference to Heidegger, Bultmann, Gadamer, and Wittgenstein. Anthony C. Thiselton , 1982 .

[15]  Karrie Karahalios,et al.  Visualizing Conversation , 1999, J. Comput. Mediat. Commun..

[16]  J. Cassell,et al.  Authorable Virtual Peers for Autism Spectrum Disorders , 2006 .

[17]  Alan E. Kazdin,et al.  Single-Case Research Designs: Methods for Clinical and Applied Settings , 2010 .

[18]  P. Mundy,et al.  Defining the social deficits of autism: the contribution of non-verbal communication measures. , 1986, Journal of child psychology and psychiatry, and allied disciplines.

[19]  Gillian R. Hayes,et al.  Designing for interaction immediacy to enhance social skills of children with autism , 2010, UbiComp.

[20]  Gillian R. Hayes,et al.  Disability studies as a source of critical inquiry for the field of assistive technology , 2010, ASSETS '10.

[21]  Peter F. Gerhardt,et al.  Addressing the Needs of Adolescents and Adults with Autism: A Crisis on the Horizon , 2011, Journal of Contemporary Psychotherapy.

[22]  Gregory D. Abowd,et al.  Interactive technologies for autism , 2007, CHI Extended Abstracts.

[23]  Lynn Dombrowski,et al.  Making "Safe": Community-Centered Practices in a Virtual World Dedicated to Children with Autism , 2015, CSCW.

[24]  Lynn Dombrowski,et al.  "Will I always be not social?": Re-Conceptualizing Sociality in the Context of a Minecraft Community for Autism , 2016, CHI.

[25]  Pattie Maes,et al.  eye-q: eyeglass peripheral display for subtle intimate notifications , 2006, Mobile HCI.

[26]  Graciela Verónica Gil Costa,et al.  Evaluating the spoken dialogue system of a conversational character: A Simulation Study , 2016 .

[27]  S. Hayes,et al.  Alternating treatments design: one strategy for comparing the effects of two treatments in a single subject. , 1979, Journal of applied behavior analysis.

[28]  Gillian R. Hayes,et al.  Would You Be Mine: Appropriating Minecraft as an Assistive Technology for Youth with Autism , 2016, ASSETS.

[29]  O. Bogdashina.  Sensory Perceptual Issues in Autism and Asperger Syndrome: Different Sensory Experiences - Different Perceptual Worlds , 2003 .

[30]  Albert A. Rizzo,et al.  Is clinical virtual reality ready for primetime? , 2017, Neuropsychology.

[31]  Nilanjan Sarkar,et al.  Understanding How Adolescents with Autism Respond to Facial Expressions in Virtual Reality Environments , 2013, IEEE Transactions on Visualization and Computer Graphics.

[32]  Christopher Frauenberger,et al.  Social Communication between Virtual Characters and Children with Autism , 2011, AIED.

[33]  Juan Ye,et al.  Wearable Assistive Technologies for Autism: Opportunities and Challenges , 2018, IEEE Pervasive Computing.

[34]  S. Mayes,et al.  Suicide Ideation and Attempts in Children with Autism. , 2013 .

[35]  Yvonne Rogers,et al.  Interaction Design - Beyond Human-Computer Interaction, 3rd Edition , 2012 .

[36]  M. Knapp,et al.  Nonverbal communication in human interaction , 1972 .

[37]  Mohammed E. Hoque,et al.  Rhema: A Real-Time In-Situ Intelligent Interface to Help People with Public Speaking , 2015, IUI.

[38]  S. Parsons,et al.  State-of-the-art of virtual reality technologies for children on the autism spectrum , 2011 .

[39]  Xinlong Jiang,et al.  ProCom: Designing and Evaluating a Mobile and Wearable System to Support Proximity Awareness for People with Autism , 2017, CHI.