Expressive visual text-to-speech as an assistive technology for individuals with autism spectrum conditions

Highlights • We show a videorealistic avatar, XpressiveTalk, which speaks in various emotional tones.• The use as an assistive technology for people with autism spectrum conditions (ASC) is discussed.• We carry out emotion-recognition and preference studies.• Adults with ASC are less accurate than controls, but above chance levels for inferring emotions.

[1]  Peter Chapman,et al.  Processing of Spontaneous Emotional Responses in Adolescents and Adults with Autism Spectrum Disorders: Effect of Stimulus Type , 2015, Autism research : official journal of the International Society for Autism Research.

[2]  S. Baron-Cohen,et al.  The “Reading the Mind in Films” Task: Complex emotion recognition in adults with and without autism spectrum conditions , 2006, Social neuroscience.

[3]  Heiga Zen,et al.  Statistical Parametric Speech Synthesis Based on Speaker and Language Factorization , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[4]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[5]  Andy P. Field,et al.  Discovering Statistics Using Ibm Spss Statistics , 2017 .

[6]  Alex Martin,et al.  Facial Emotion Recognition in Autism Spectrum Disorders: A Review of Behavioral and Neuroimaging Studies , 2010, Neuropsychology Review.

[7]  H. Roeyers,et al.  Advancing advanced mind-reading tests: empathic accuracy in adults with a pervasive developmental disorder. , 2001, Journal of child psychology and psychiatry, and allied disciplines.

[8]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Sarah-Jayne Blakemore,et al.  Brief Report: Perception of Genuine and Posed Smiles by Individuals with Autism , 2008, Journal of autism and developmental disorders.

[10]  S. Baron-Cohen,et al.  The Autism-Spectrum Quotient (AQ): Evidence from Asperger Syndrome/High-Functioning Autism, Malesand Females, Scientists and Mathematicians , 2001, Journal of autism and developmental disorders.

[11]  S. Baron-Cohen,et al.  Enhancing Emotion Recognition in Children with Autism Spectrum Conditions: An Intervention Using Animated Vehicles with Real Emotional Faces , 2010, Journal of autism and developmental disorders.

[12]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[13]  G. Dawson,et al.  Early social attention impairments in autism: social orienting, joint attention, and attention to distress. , 2004, Developmental psychology.

[14]  D. Skuse,et al.  Avoidance of emotionally arousing stimuli predicts social–perceptual impairment in Asperger's syndrome , 2008, Neuropsychologia.

[15]  Gnanathusharan Rajendran,et al.  Virtual environments and autism: a developmental psychopathological approach , 2013, J. Comput. Assist. Learn..

[16]  N. Minshew,et al.  A fine-grained analysis of facial expression processing in high-functioning adults with autism , 2007, Neuropsychologia.

[17]  M. Larson.  Analysis of Variance , 2008, Circulation.

[18]  S. Baron-Cohen.  Mindblindness: An Essay on Autism and Theory of Mind , 1997 .

[19]  F. Volkmar,et al.  The enactive mind, or from actions to cognition: lessons from autism. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[20]  Sebastian B. Gaigg,et al.  The Interplay between Emotion and Cognition in Autism Spectrum Disorder: Implications for Developmental Theory , 2012, Front. Integr. Neurosci..

[21]  L. Kanner.  Autistic disturbances of affective contact. , 1968, Acta paedopsychiatrica.

[22]  J. Piven,et al.  Abnormal Processing of Social Information from Faces in Autism , 2001, Journal of Cognitive Neuroscience.

[23]  Andy P. Field,et al.  Discovering Statistics Using SPSS , 2000 .

[24]  Antonia Hamilton,et al.  Recognition of Emotions in Autism: A Formal Meta-Analysis , 2013, Journal of autism and developmental disorders.

[25]  C. Deruelle,et al.  Typical Emotion Processing for Cartoon but not for Real Faces in Children with Autistic Spectrum Disorders , 2008, Journal of autism and developmental disorders.

[26]  G. Baranek.  Autism During Infancy: A Retrospective Video Analysis of Sensory-Motor and Social Behaviors at 9–12 Months of Age , 1999, Journal of autism and developmental disorders.

[27]  S. Baron-Cohen,et al.  Systemizing empathy: Teaching adults with Asperger syndrome or high-functioning autism to recognize complex emotions using interactive multimedia , 2006, Development and Psychopathology.

[28]  N. Rinehart,et al.  Emotion recognition of static and dynamic faces in autism spectrum disorder , 2014, Cognition & emotion.

[29]  Jessica H. Schroeder,et al.  The McGurk Effect in Children With Autism and Asperger Syndrome , 2014, Autism research : official journal of the International Society for Autism Research.

[30]  D. Massaro,et al.  Visual-auditory integration during speech imitation in autism. , 2004, Research in developmental disabilities.

[31]  D. Matsumoto,et al.  Cross-Cultural Judgments of Spontaneous Facial Expressions of Emotion , 2009 .

[32]  S. Baron-Cohen.  Autism: The Empathizing–Systemizing (E‐S) Theory , 2009, Annals of the New York Academy of Sciences.

[33]  Björn Stenger,et al.  Expressive Visual Text-to-Speech Using Active Appearance Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  A. E. Cook,et al.  Face processing in children with autism , 2007, Autism : the international journal of research and practice.

[35]  P. Chapman,et al.  Do Gaze Cues in Complex Scenes Capture and Direct the Attention of High Functioning Adolescents with ASD? Evidence from Eye-tracking , 2010, Journal of autism and developmental disorders.

[36]  P. Mitchell,et al.  Using Other Minds as a Window Onto the World: Guessing What Happened from Clues in Behaviour , 2014, Journal of autism and developmental disorders.

[37]  P. Chapman,et al.  Can Adults With Autism Spectrum Disorders Infer What Happened to Someone From Their Emotional Response? , 2014, Autism research : official journal of the International Society for Autism Research.

[38]  Michael C. Frank,et al.  Eye-movements reveal attention to social information in autism spectrum disorder , 2009, Neuropsychologia.

[39]  B. Scassellati,et al.  Robots for use in autism research. , 2012, Annual review of biomedical engineering.

[40]  F. Volkmar,et al.  Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. , 2002, Archives of general psychiatry.

[41]  Heiga Zen,et al.  Statistical Parametric Speech Synthesis , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[43]  Rosalind W. Picard,et al.  Acted vs. natural frustration and delight: Many people smile in natural frustration , 2011, Face and Gesture 2011.

[44]  R. Fukuda,et al.  fMRI study of recognition of facial expressions in high-functioning autistic patients , 2003, Neuroreport.

[45]  John M. Findlay,et al.  Brief Report: Young Adults with Autism Spectrum Disorder Show Normal Attention to Eye-Gaze Information—Evidence from a New Change Blindness Paradigm , 2008, Journal of autism and developmental disorders.

[46]  Mark J. F. Gales,et al.  Speech factorization for HMM-TTS based on cluster adaptive training , 2012, INTERSPEECH.

[47]  D. Perrett,et al.  Detecting subtle facial emotion recognition deficits in high-functioning Autism using dynamic stimuli of varying intensities , 2010, Neuropsychologia.

[48]  Keiichi Tokuda,et al.  Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis , 1999, EUROSPEECH.

[49]  K. O’Connor.  Brief Report: Impaired Identification of Discrepancies Between Expressive Faces and Voices in Adults with Asperger’s Syndrome , 2007, Journal of autism and developmental disorders.

[50]  N. Minshew,et al.  Misinterpretation of facial expressions of emotion in verbal adults with autism spectrum disorder , 2015, Autism : the international journal of research and practice.

[51]  Marcello Ferro,et al.  Human-Robot Interaction in Autism: FACE, an Android-based Social Therapy , 2007, RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication.

[52]  G. Dawson,et al.  Early recognition of children with autism: A study of first birthday home videotapes , 1994, Journal of autism and developmental disorders.

[53]  Yong Tao,et al.  Compound facial expressions of emotion , 2014, Proceedings of the National Academy of Sciences.

[54]  B. Ingersoll,et al.  The use of innovative computer technology for teaching social communication to individuals with autism spectrum disorders , 2011 .

[55]  P. Chapman,et al.  The eye gaze direction of an observed person can bias perception, memory, and attention in adolescents with and without autism spectrum disorder. , 2010, Journal of experimental child psychology.