Facial expression recognition teaching to preschoolers with autism: a natural user interface approach

The recognition of facial expressions is important for the perception of emotions. Understanding emotions is essential in human communication and social interaction. Children with autism have been reported to exhibit deficits in the recognition of affective expressions. Their difficulties in understanding and expressing emotions lead to inappropriate behavior derived from their inability to interact adequately with other people. Those deficits seem to be rather permanent in individuals with autism so intervention tools for improving those impairments are desirable. Educational interventions for teaching emotion recognition from facial expressions should occur as early as possible in order to be successful and to have a positive effect. It is claimed that Serious Games can be very effective in the areas of therapy and education for children with autism. However, those computer interventions require considerable skills for interaction. Before the age of 6, most children with autism do not have such basic motor skills in order to manipulate a mouse or a keyboard. Our approach takes account of the specific characteristics of preschoolers with autism and their physical inabilities. By creating an educational computer game, which provides physical interaction by employing natural user interface (NUI), we aim to support early intervention and to foster facial expression learning.

[1]  Ofer Golan,et al.  Transported to a World of Emotion , 2007, McGill journal of medicine : MJM : an international forum for the advancement of medical sciences by students.

[2]  B. Tonge,et al.  Teaching emotion recognition skills to young children with autism: a randomised controlled trial of an emotion training programme. , 2012, Journal of child psychology and psychiatry, and allied disciplines.

[3]  I. Scott MacKenzie,et al.  Accuracy measures for evaluating computer pointing devices , 2001, CHI.

[4]  Temple Grandin,et al.  Thinking in Pictures: And Other Reports From My Life With Autism , 1996 .

[5]  G. Schwarzer,et al.  Face processing in 8-month-old infants: evidence for configural and analytical processing , 2003, Vision Research.

[6]  N. Bauminger,et al.  The Facilitation of Social-Emotional Understanding and Social Interaction in High-Functioning Children with Autism: Intervention Outcomes , 2002, Journal of autism and developmental disorders.

[7]  Family Connections: Visual Supports for Promoting Social Skills in Young Children: A Family Perspective , 2012 .

[8]  Bretagne Abirached,et al.  Understanding User Needs for Serious Games for Teaching Children with Autism Spectrum Disorders Emotions , 2012 .

[9]  Marc H. Bornstein,et al.  Recognition, discrimination and categorization of smiling by 5‐month‐old infants , 2003 .

[10]  S. Baron-Cohen,et al.  Systemizing empathy: Teaching adults with Asperger syndrome or high-functioning autism to recognize complex emotions using interactive multimedia , 2006, Development and Psychopathology.

[11]  Gondy Leroy,et al.  A Smart-Phone Application and a Companion Website for the Improvement of the Communication Skills of Children with Autism: Clinical Rationale, Technical Development and Preliminary Results , 2011, Journal of Medical Systems.

[12]  R. Hobson The autistic child's appraisal of expressions of emotion. , 1986, Journal of child psychology and psychiatry, and allied disciplines.

[13]  S. Baron-Cohen,et al.  Enhanced visual search for a conjunctive target in autism: a research note. , 1998, Journal of child psychology and psychiatry, and allied disciplines.

[14]  B. Ingersoll,et al.  The use of innovative computer technology for teaching social communication to individuals with autism spectrum disorders , 2011 .

[15]  J. Stockman,et al.  Absence of Preferential Looking to the Eyes of Approaching Adults Predicts Level of Social Disability in 2-Year-Old Toddlers With Autism Spectrum Disorder , 2010 .

[16]  R. Schultz,et al.  The perception and identification of facial emotions in individuals with autism spectrum disorders using the Let's Face It! Emotion Skills Battery. , 2012, Journal of child psychology and psychiatry, and allied disciplines.

[17]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[18]  S. Baron-Cohen,et al.  Enhancing Emotion Recognition in Children with Autism Spectrum Conditions: An Intervention Using Animated Vehicles with Real Emotional Faces , 2010, Journal of autism and developmental disorders.

[19]  James P. Morris,et al.  Multimodal emotion processing in autism spectrum disorders: An event-related potential study , 2013, Developmental Cognitive Neuroscience.

[20]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[21]  U. Frith,et al.  Brief Report: Cognitive Processing of Own Emotions in Individuals with Autistic Spectrum Disorder and in Their Relatives , 2004, Journal of autism and developmental disorders.

[22]  Nopporn Chotikakamthorn,et al.  Using tangible user interfaces in computer-based training systems for low-functioning autistic children , 2011, Personal and Ubiquitous Computing.

[23]  I. Scott MacKenzie,et al.  ISO 9241-9 evaluation of video game controllers , 2009, Graphics Interface.

[24]  Byron Wine,et al.  APPLIED BEHAVIOR ANALYSIS: AUTISM AND BEYOND , 2012 .

[25]  Emma Ashwin,et al.  Can emotion recognition be taught to children with autism spectrum conditions? , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[26]  N. C. Pee,et al.  Serious Game for Autism Children: Review of Literature , 2012 .

[27]  M. Wagner,et al.  Auditory hypersensitivity in the autistic spectrum disorder. , 2008, Pro-fono : revista de atualizacao cientifica.

[28]  S. Baron-Cohen,et al.  Eagle-Eyed Visual Acuity: An Experimental Investigation of Enhanced Perception in Autism , 2009, Biological Psychiatry.

[29]  Lawrence Scahill,et al.  Social Skills Development in Children with Autism Spectrum Disorders: A Review of the Intervention Research , 2007, Journal of autism and developmental disorders.

[30]  S. Bölte,et al.  Emotion Recognition in Children and Adolescents with Autism Spectrum Disorders , 2009, Journal of autism and developmental disorders.

[31]  J. Baio Prevalence of autism spectrum disorders--Autism and Developmental Disabilities Monitoring Network, 14 sites, United States, 2008. , 2012, Morbidity and mortality weekly report. Surveillance summaries.

[32]  Alison E. Lane,et al.  Factors influencing skilled use of the computer mouse by school-aged children , 2010, Comput. Educ..

[33]  S. Baron-Cohen,et al.  Superior visual search in autism. , 2001, Journal of experimental psychology. Human perception and performance.

[34]  R. Hastings,et al.  How and why do parents choose Early Intensive Behavioral Intervention for their young child with Autism , 2012 .

[35]  S. Rogers,et al.  Brief report: Early intervention in autism , 1996, Journal of autism and developmental disorders.

[36]  Marian Stewart Bartlett,et al.  Emotion Mirror: A Novel Intervention for Autism Based on Real-Time Expression Recognition , 2012, ECCV Workshops.

[37]  J. Farber,et al.  Understanding Autism , 1984, Clinical pediatrics.

[38]  Katherine B. Martin,et al.  Facial Action Coding System , 2015 .

[39]  Bretagne Abirached,et al.  A framework for designing assistive technologies for teaching children with ASDs emotions , 2012, CHI Extended Abstracts.

[40]  Brian Reichow,et al.  Overview of Meta-Analyses on Early Intensive Behavioral Intervention for Young Children with Autism Spectrum Disorders , 2011, Journal of Autism and Developmental Disorders.

[41]  C. Gray,et al.  Social Stories: Improving Responses of Students with Autism with Accurate Social Information , 1993 .

[42]  Tiffany Barnes,et al.  cMotion: A New Game Design to Teach Emotion Recognition and Programming Logic to Children using Virtual Humans , 2009, 2009 IEEE Virtual Reality Conference.

[43]  K. Pajareya,et al.  A one-year prospective follow-up study of a DIR/Floortime parent training intervention for pre-school children with autistic spectrum disorders. , 2012, Journal of the Medical Association of Thailand = Chotmaihet thangphaet.

[44]  E. Fombonne,et al.  Sensori-motor and Daily Living Skills of Preschool Children with Autism Spectrum Disorders , 2009, Journal of autism and developmental disorders.

[45]  C. Gray,et al.  The New Social Story Book , 1994 .

[46]  Garrison W. Cottrell,et al.  Transmitting and Decoding Facial Expressions , 2005, Psychological science.

[47]  Sarah A. Douglas,et al.  Testing pointing device performance and user assessment with the ISO 9241, Part 9 standard , 1999, CHI '99.

[48]  H. Wellman,et al.  Early understanding of emotion: Evidence from natural language , 1995 .

[49]  K. Durand,et al.  The development of facial emotion recognition: the role of configural information. , 2007, Journal of experimental child psychology.

[50]  N. Minshew,et al.  The development of emotion recognition in individuals with autism. , 2009, Child development.

[51]  Rana El Kaliouby,et al.  An exploratory social-emotional prosthetic for autism spectrum disorders , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[52]  J. Ganz,et al.  The Picture Exchange Communication System (PECS): A Promising Method for Improving Communication Skills of Learners with Autism Spectrum Disorders. , 2012 .

[53]  Tiago Fernandes,et al.  Interactive Technology: Teaching People with Autism to Recognize Facial Emotions , 2011 .

[54]  Chun-Yan Xi,et al.  Auditory abnormalities in children with autism , 2012 .

[55]  I. Scott MacKenzie,et al.  Changes in applied force to a touchpad during pointing tasks , 2002 .

[56]  J. Baio Morbidity and Mortality Weekly Report Prevalence of Autism Spectrum Disorders — Autism and Developmental Disabilities Monitoring Network, Six Sites, United States, 2000; Prevalence of Autism Spectrum Disorders — Autism and Developmental Disabilities Monitoring Network, 14 Sites, United States, 2002; , 2007 .

[57]  Y. S. Kim,et al.  Global Prevalence of Autism and Other Pervasive Developmental Disorders , 2012, Autism research : official journal of the International Society for Autism Research.

[58]  Aubyn C. Stahmer,et al.  Inclusion for toddlers with autism spectrum disorders , 2011, Autism : the international journal of research and practice.

[59]  B. Prizant,et al.  The SCERTS Model: A Transactional, Family‐Centered Approach to Enhancing Communication and Socioemotional Abilities of Children With Autism Spectrum Disorder , 2003 .

[60]  J. Piven,et al.  Visual Scanning of Faces in Autism , 2002, Journal of autism and developmental disorders.

[61]  Geraldine Dawson,et al.  Becoming a Face Expert: A Computerized Face-Training Program for High-Functioning Individuals With Autism Spectrum Disorders , 2007, Developmental neuropsychology.

[62]  S. Eikeseth Outcome of comprehensive psycho-educational interventions for young children with autism. , 2009, Research in developmental disabilities.

[63]  Francisco Javier Ballesta Pagán,et al.  Software para enseñar emociones al alumnado con trastorno del espectro autista , 2011 .

[64]  Alex Martin,et al.  Facial Emotion Recognition in Autism Spectrum Disorders: A Review of Behavioral and Neuroimaging Studies , 2010, Neuropsychology Review.

[65]  James W Tanaka,et al.  Using computerized games to teach face recognition skills to children with autism spectrum disorder: the Let's Face It! program. , 2010, Journal of child psychology and psychiatry, and allied disciplines.

[66]  Antonia Hamilton,et al.  Recognition of Emotions in Autism: A Formal Meta-Analysis , 2013, Journal of autism and developmental disorders.

[67]  Paul T. Sowden,et al.  Color Perception in Children with Autism , 2008, Journal of autism and developmental disorders.

[68]  Konstantin Mitgutsch,et al.  Purposeful by design?: a serious game design assessment framework , 2012, FDG.

[69]  M. Silver,et al.  Evaluation of a New Computer Intervention to Teach People with Autism or Asperger Syndrome to Recognize and Predict Emotions in Others , 2001, Autism : the international journal of research and practice.

[70]  Paul T. Sowden,et al.  Reduced chromatic discrimination in children with autism spectrum disorders. , 2010, Developmental science.

[71]  Jean-Claude Martin,et al.  Multimedia interfaces for users with high functioning autism: An empirical investigation , 2008, Int. J. Hum. Comput. Stud..

[72]  L. LeBlanc,et al.  Use of Technology in Interventions for Children with Autism , 2004 .

[73]  Jessica L. Tracy,et al.  Is Emotion Recognition Impaired in Individuals with Autism Spectrum Disorders? , 2010, Journal of autism and developmental disorders.

[74]  E. Schopler,et al.  The Teacch Approach to Autism Spectrum Disorders , 2004, Springer US.

[75]  P. J. Brooks,et al.  Use of Computer-Assisted Technologies (CAT) to Enhance Social, Communicative, and Language Development in Children with Autism Spectrum Disorders , 2012, Journal of Autism and Developmental Disorders.

[76]  J. Russell,et al.  Children acquire emotion categories gradually , 2008 .

[77]  R. Coulter Understanding the Visual Symptoms of Individuals with Autism Spectrum Disorder (ASD) , 2009 .