Rachel: Design of an emotionally targeted interactive agent for children with autism

Increasingly, multimodal human-computer interactive tools are leveraged in both autism research and therapies. Embodied conversational agents (ECAs) are employed to facilitate the collection of socio-emotional interactive data from children with autism. In this paper we present an overview of the Rachel system developed at the University of Southern California. The Rachel ECA is designed to elicit and analyze complex, structured, and naturalistic interactions and to encourage affective and social behavior. The pilot studies suggest that this tool can be used to effectively elicit social conversational behavior. This paper presents a description of the multimodal human-computer interaction system and an overview of the collected data. Future work includes utilizing signal processing techniques to provide a quantitative description of the interaction patterns.

[1]  Shrikanth S. Narayanan,et al.  Automatic Detection of Disfluency Boundaries in Spontaneous Speech of Children Using Audio–Visual Information , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[2]  Justine Cassell,et al.  Playing with virtual peers: bootstrapping contingent discourse in children with autism , 2008, ICLS.

[3]  Shrikanth S. Narayanan,et al.  An empirical analysis of user uncertainty in problem-solving child-machine interactions , 2008, WOCCI.

[4]  Jennifer Healey,et al.  Toward Machine Emotional Intelligence: Analysis of Affective Physiological State , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Shrikanth S. Narayanan,et al.  Comparison of child-human and child-computer interactions based on manual annotations , 2009, WOCCI '09.

[6]  Matthew S. Goodwin,et al.  Cardiovascular Arousal in Individuals With Autism , 2006 .

[7]  Shrikanth S. Narayanan,et al.  Detecting emotional state of a child in a conversational computer game , 2011, Comput. Speech Lang..

[8]  R. Landa.  Early communication development and intervention for children with autism. , 2007, Mental retardation and developmental disabilities research reviews.

[9]  Shrikanth S. Narayanan,et al.  Creating conversational interfaces for children , 2002, IEEE Trans. Speech Audio Process..

[10]  Justine Cassell,et al.  Running head: Virtual Peer Technology for Children with Autism Using Virtual Peer Technology as an Intervention for Children with Autism , 2006 .

[11]  Luis A. Hernández Gómez,et al.  A collaborative approach to the design and evaluation of an interactive learning tool for children with special educational needs , 2010, IDC.

[12]  Yonghong Yan,et al.  Universal speech tools: the CSLU toolkit , 1998, ICSLP.

[13]  Ying Liu,et al.  A Multilingual Embodied Conversationa Language , 2006 .

[14]  Gérard Bailly,et al.  Gaze, conversational agents and face-to-face communication , 2010, Speech Commun..

[15]  J. Rabe-Jabłońska,et al.  [Affective disorders in the fourth edition of the classification of mental disorders prepared by the American Psychiatric Association -- diagnostic and statistical manual of mental disorders]. , 1993, Psychiatria polska.

[16]  Shrikanth S. Narayanan,et al.  Reference marking in children's computer-directed speech: an integrated analysis of discourse and gestures , 2004, INTERSPEECH.