Measuring Engagement in Robot-Assisted Autism Therapy: A Cross-Cultural Study

During occupational therapy for children with autism, it is often necessary to elicit and maintain engagement for the children to benefit from the session. Recently, social robots have been used for this; however, existing robots lack the ability to autonomously recognize the children's level of engagement, which is necessary when choosing an optimal interaction strategy. Progress in automated engagement reading has been impeded in part due to a lack of studies on child-robot engagement in autism therapy. While there are large individual differences within the autistic population, little is known about variation across cultures. We analyzed the engagement of children (age 3-13) from two different cultural backgrounds: Asia (Japan, n=17) and Eastern Europe (Serbia, n=19). The children participated in a 25 minute therapy session during which we studied the relationship between the children's behavioural engagement (task-driven) and different facets of affective engagement (valence and arousal). Although our results indicate that there are statistically significant differences in engagement displays in the two groups, it is difficult to make any causal claims about these differences due to the large variation in age and behavioural severity of the children in the study. However, our exploratory analysis reveals important associations between target engagement and perceived levels of valence and arousal, indicating that these can be used as a proxy for the children's engagement during the therapy. We provide suggestions on how this can be leveraged to optimize social robots for autism therapy, while taking into account cultural differences.

[1]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .

[2]  Michael J. Richardson,et al.  Rhythmic bimanual coordination is impaired in young children with autism spectrum disorder , 2012 .

[3]  Bob Algozzine,et al.  Multicultural Issues in Autism , 2004, Journal of autism and developmental disorders.

[4]  B. Robins,et al.  Does appearance matter in the interaction of children with autism with a humanoid robot , 2006 .

[5]  S. Baron-Cohen,et al.  Teaching Children with Autism to Mind-Read: A Practical Guide for Teachers and Parents , 1998 .

[6]  E. Mirand,et al.  Research and treatment , 1983 .

[7]  Kerstin Dautenhahn,et al.  The effectiveness of using a robotics class to foster collaboration among groups of children with autism in an exploratory study , 2010, Personal and Ubiquitous Computing.

[8]  Cynthia Breazeal,et al.  Affective Interaction between Humans and Robots , 2001, ECAL.

[9]  P. Ekman,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .

[10]  D. Bailey,et al.  Effects of Classroom Social Structure and Disability on Engagement , 1995 .

[11]  Thomas F Gross,et al.  The Perception of Four Basic Emotions in Human and Nonhuman Faces by Children With Autism and Other Developmental Disabilities , 2004, Journal of abnormal child psychology.

[12]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[13]  Catherine J. Stevens,et al.  Continuous self-report of engagement to live solo marimba performance , 2008 .

[14]  L. Y. Lo,et al.  Angry Versus Furious: A Comparison Between Valence and Arousal in Dimensional Models of Emotions , 2016, The Journal of psychology.

[15]  Hatice Gunes,et al.  Multimodal Affect Recognition for Naturalistic Human-Computer and Human-Robot Interactions , 2015 .

[16]  Sara E. Rimm-Kaufman,et al.  Kindergarten Classroom Quality, Behavioral Engagement, and Reading Achievement , 2009 .

[17]  W. Watts Relative persistence of opinion change induced by active compared to passive participation. , 1967, Journal of personality and social psychology.

[18]  Hatice Gunes,et al.  Fully Automatic Analysis of Engagement and Its Relationship to Personality in Human-Robot Interactions , 2017, IEEE Access.

[19]  Andrea C. Pierno,et al.  Robotic movement elicits visuomotor priming in children with autism , 2008, Neuropsychologia.

[20]  Nangyeon Lim,et al.  Cultural differences in emotion: differences in emotional arousal level between the East and the West , 2016, Integrative medicine research.

[21]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  H. Kozima,et al.  Children-robot interaction: a pilot study in autism therapy. , 2007, Progress in brain research.

[23]  Tamara C Daley,et al.  The Need for Cross-Cultural Research on the Pervasive Developmental Disorders , 2002 .

[24]  Ayanna M. Howard,et al.  Identifying Engagement from Joint Kinematics Data for Robot Therapy Prompt Interventions for Children with Autism Spectrum Disorder , 2016, ICSR.

[25]  Li-Jun Ji,et al.  Two Sides of Emotion: Exploring Positivity and Negativity in Six Basic Emotions across Cultures , 2017, Front. Psychol..

[26]  Peter H. Kahn,et al.  Robotic animals might aid in the social development of children with autism , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  K. Scherer,et al.  Evidence for universality and cultural variation of differential emotion response patterning. , 1994, Journal of personality and social psychology.

[28]  Cipresso Pietro,et al.  The Pursuit of Happiness Measurement: A Psychometric Model Based on Psychophysiological Correlates , 2014, TheScientificWorldJournal.

[29]  C. Crowell,et al.  The Clinical Use of Robots for Individuals with Autism Spectrum Disorders: A Critical Review. , 2012, Research in autism spectrum disorders.

[30]  Eva Beckung,et al.  Developmental coordination disorder and other motor control problems in girls with autism spectrum disorder and/or attention-deficit/hyperactivity disorder. , 2010, Research in developmental disabilities.

[31]  Matthew S. Goodwin,et al.  Harnessing Repetitive Behaviours to Engage Attention and Learning in a Novel Therapy for Autism: An Exploratory Analysis , 2012, Front. Psychology.

[32]  K. Dautenhahn,et al.  Towards interactive robots in autism therapy: background, motivation and challenges , 2004 .

[33]  J. Engelmann,et al.  Emotion perception across cultures: the role of cognitive mechanisms , 2012, Front. Psychol..

[34]  Kerstin Dautenhahn,et al.  A Pilot Study with a Novel Setup for Collaborative Play of the Humanoid Robot KASPAR with Children with Autism , 2014, Int. J. Soc. Robotics.

[35]  Patrick Bateson,et al.  The Mechanisms of Imitation , 2001, Cybern. Syst..

[36]  J. Russell,et al.  A Description of the Affective Quality Attributed to Environments , 1980 .

[37]  Geoffrey Bird,et al.  Can Neurotypical Individuals Read Autistic Facial Expressions? Atypical Production of Emotional Facial Expressions in Autism Spectrum Disorders , 2015, Autism research : official journal of the International Society for Autism Research.

[38]  Helen McConachie,et al.  Diagnosing Autism Spectrum Disorders in Pre-school Children Using Two Standardised Assessment Instruments: The ADI-R and the ADOS , 2008, Journal of autism and developmental disorders.

[39]  Hillary Anger Elfenbein,et al.  On the universality and cultural specificity of emotion recognition: a meta-analysis. , 2002, Psychological bulletin.

[40]  B. Scassellati,et al.  Robots for use in autism research. , 2012, Annual review of biomedical engineering.

[41]  J. Russell Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. , 1994, Psychological bulletin.

[42]  Cheryl Thacker Handbook on Autism. , 1996 .

[43]  Agata Rozga,et al.  Increased Eye Contact During Conversation Compared to Play in Children With Autism , 2017, Journal of autism and developmental disorders.

[44]  Fabio Tesser,et al.  Multimodal child-robot interaction: building social bonds , 2013, HRI 2013.

[45]  Utku Şentürk,et al.  Erratum to “Does Urinary Bladder Shape Affect Urinary Flow Rate in Men with Lower Urinary Tract Symptoms?” , 2014, The Scientific World Journal.

[46]  G. Arbanas Diagnostic and Statistical Manual of Mental Disorders (DSM-5) , 2015 .

[47]  Stephanie Kitzhaber Interventions for Multicultural Children with Autism , 2012 .

[48]  P. Ravindra De Silva,et al.  Therapeutic-assisted robot for children with autism , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[49]  J. Connell Context, self, and action: A motivational analysis of self-system processes across the life span. , 1990 .

[50]  Tom Ziemke,et al.  Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook , 2012, Paladyn J. Behav. Robotics.

[51]  Illah R. Nourbakhsh,et al.  A survey of socially interactive robots , 2003, Robotics Auton. Syst..

[52]  M. Wiznitzer,et al.  AIDS and the Nervous System , 1988, Neurology.

[53]  Rosalind W. Picard Future affective technology for autism and emotion communication , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[54]  Edward G. Carmines,et al.  Reliability and Validity Assessment , 1979 .

[55]  Alexander Libin,et al.  Person-robot interactions from the robopsychologists' point of view: the robotic psychology and robotherapy approach , 2004, Proceedings of the IEEE.

[56]  Tony Charman,et al.  Screening and surveillance for autism and pervasive developmental disorders , 2022 .

[57]  Marc Schneider,et al.  Quantitative Assessment of Neuromotor Function in Adolescents with High Functioning Autism and Asperger Syndrome , 2007, Journal of autism and developmental disorders.

[58]  Yi Zhang,et al.  Motor impairment in sibling pairs concordant and discordant for autism spectrum disorders , 2012, Autism : the international journal of research and practice.

[59]  Michael J. Richardson,et al.  Autism and social disconnection in interpersonal rocking , 2013, Front. Integr. Neurosci..

[60]  Brian Scassellati,et al.  Bridging the research gap , 2012, HRI 2012.

[61]  Kerstin Dautenhahn,et al.  Human-centred design methods: Developing scenarios for robot assisted play informed by user panels and field trials , 2010, Int. J. Hum. Comput. Stud..

[62]  Takayuki Kanda,et al.  > Replace This Line with Your Paper Identification Number (double-click Here to Edit) < 2 , 2022 .

[63]  Kerstin Dautenhahn,et al.  Developing scenarios for robot assisted play , 2008, RO-MAN 2008 - The 17th IEEE International Symposium on Robot and Human Interactive Communication.

[64]  Maki K. Habib Handbook of Research on Advancements in Robotics and Mechatronics , 2014 .

[65]  Agata Rozga,et al.  Using electrodermal activity to recognize ease of engagement in children during social interactions , 2014, UbiComp.

[66]  S. Mostofsky,et al.  Dyspraxia in autism: association with motor, social, and communicative deficits , 2007, Developmental medicine and child neurology.

[67]  Mary Ainley,et al.  Affective engagement: A person-centred approach to understanding the structure of subjective learning experiences. , 2004 .

[68]  Prithvi Perepa,et al.  Cultural basis of social ‘deficits’ in autism spectrum disorders , 2014 .

[69]  Brian Scassellati,et al.  How Social Robots Will Help Us to Diagnose, Treat, and Understand Autism , 2005, ISRR.

[70]  Catherine J. Stevens,et al.  A Continuous measure of musical engagement contributes to prediction of perceived arousal and valence , 2014 .

[71]  Cynthia Breazeal,et al.  Emotion and sociable humanoid robots , 2003, Int. J. Hum. Comput. Stud..

[72]  Hanna Damasio,et al.  Cultural modes of expressing emotions influence how emotions are experienced. , 2016, Emotion.

[73]  Jaeryoung Lee,et al.  NAO-Dance Therapy for Children with ASD , 2017, HRI.

[74]  Matthew S. Goodwin,et al.  MEASURING AUTONOMIC AROUSAL DURING THERAPY , 2012 .

[75]  Hatice Gunes,et al.  Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space , 2011, IEEE Transactions on Affective Computing.

[76]  John-Jules Ch. Meyer,et al.  Adaptive Emotional Expression in Robot-Child Interaction , 2014, 2014 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[77]  Mark Carter,et al.  The effect of activity type on the engagement and interaction of young children with disabilities in inclusive childcare settings , 2013 .

[78]  Coral Kemp,et al.  The Engagement and Interaction of Children With Autism Spectrum Disorder in Segregated and Inclusive Early Childhood Center-Based Settings , 2009 .

[79]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[80]  Mohamed Chetouani,et al.  Engagement detection based on mutli-party cues for human robot interaction , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[81]  F. Happé,et al.  Time to give up on a single explanation for autism , 2006, Nature Neuroscience.

[82]  Pierre Dillenbourg,et al.  From real-time attention assessment to “with-me-ness” in human-robot interaction , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[83]  Sofiane Boucenna,et al.  Evaluating the Engagement with Social Robots , 2015, International Journal of Social Robotics.

[84]  Changchun Liu,et al.  Physiology-based affect recognition for computer-assisted intervention of children with Autism Spectrum Disorder , 2008, Int. J. Hum. Comput. Stud..

[85]  Welch Bl THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[86]  D. Keen,et al.  Engagement of Children With Autism in Learning , 2009 .

[87]  S. Baron-Cohen,et al.  Does the autistic child have a “theory of mind” ? , 1985, Cognition.

[88]  K. Dautenhahn,et al.  A long-term study of children with autism playing with a robotic pet: Taking inspirations from non-directive play therapy to encourage children’s proactivity and initiative-taking , 2009 .

[89]  Alessandro G. Di Nuovo,et al.  A cross-cultural study of acceptance and use of robotics by future psychology practitioners , 2015, 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[90]  Connie Kasari,et al.  Play and Joint Attention of Children with Autism in the Preschool Special Education Classroom , 2012, Journal of Autism and Developmental Disorders.

[91]  Björn W. Schuller,et al.  Emotion representation, analysis and synthesis in continuous space: A survey , 2011, Face and Gesture 2011.

[92]  Cynthia Breazeal,et al.  Designing sociable robots , 2002 .

[93]  C. Hass,et al.  Motor Coordination in Autism Spectrum Disorders: A Synthesis and Meta-Analysis , 2010, Journal of autism and developmental disorders.

[94]  Aude Billard,et al.  Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? , 2005, Universal Access in the Information Society.

[95]  Dirk Adolph,et al.  Valence and arousal: a comparison of two sets of emotional facial expressions. , 2010, The American journal of psychology.

[96]  山田 幸恵,et al.  Human-Robot Interaction (HRI) における人の態度・不安・行動 , 2010 .

[97]  J. Meece,et al.  Students' goal orientations and cognitive engagement in classroom activities , 1988 .

[98]  Matt Tincani,et al.  Race, Culture, and Autism Spectrum Disorder: Understanding the Role of Diversity in Successful Educational Interventions , 2009 .

[99]  James J. Appleton,et al.  Measuring cognitive and psychological engagement: Validation of the Student Engagement Instrument , 2006 .

[100]  Mohamed Chetouani,et al.  A multi-level context-based modeling of engagement in Human-Robot Interaction , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[101]  Maja Pantic,et al.  Sentiment apprehension in human-robot interaction with NAO , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[102]  Rana el Kaliouby Large-scale observational evidence of cross-cultural differences in facial behavior , 2018, BDJ.

[103]  Shinobu Kitayama,et al.  Cultural constructions of happiness: theory and emprical evidence , 2004 .

[104]  David S Mandell,et al.  The role of culture in families' treatment decisions for children with autism spectrum disorders. , 2005, Mental retardation and developmental disabilities research reviews.

[105]  M. A. Cascio,et al.  Cross-Cultural Autism Studies, Neurodiversity, and Conceptualizations of Autism , 2015, Culture, medicine and psychiatry.

[106]  Giacomo Vivanti,et al.  Mechanisms of Imitation Impairment in Autism Spectrum Disorder , 2014, Journal of abnormal child psychology.

[107]  Ana Paiva,et al.  Context-Sensitive Affect Recognition for a Robotic Game Companion , 2014, ACM Trans. Interact. Intell. Syst..

[108]  S. Odom,et al.  Widening the Circle: Including Children with Disabilities in Preschool Programs. Early Childhood Education Series. , 2001 .

[109]  A. J. Fridlund,et al.  Facial Expressions , 2018, Encyclopedia of Evolutionary Psychological Science.