Detecting Politeness and frustration state of a child in a conversational computer game

In this study, we investigate politeness and frustration behavior of children during their spoken interaction with computer characters in a game. We focus on automatically detecting frustrated, polite and neutral attitudes from the child’s speech (acoustic and language) communication cues and study their differences as a function of age and gender. The study is based on a Wizard-of-Oz dialog corpus of 103 children playing a voice activated computer game. Statistical analysis revealed that there was a significant gender effect on politeness with girls in this data exhibiting more explicit politeness markers. The analysis also showed that there is a positive correlation between frustration and the number of dialog turns reflecting the fact that longer time spent solving the puzzle of the game led to a more frustrated child. By combining acoustic and language cues for the task of automatic detection of politeness and frustration, we obtain average accuracy of 84.7% and 71.3%, respectively, by using age dependent models and 85% and 72%, respectively, for gender dependent models.

[1]  K. Fischer,et al.  DESPERATELY SEEKING EMOTIONS OR: ACTORS, WIZARDS, AND HUMAN BEINGS , 2000 .

[2]  Shrikanth S. Narayanan,et al.  Creating conversational interfaces for children , 2002, IEEE Trans. Speech Audio Process..

[3]  Diane J. Litman,et al.  Predicting Student Emotions in Computer-Human Tutoring Dialogues , 2004, ACL.

[4]  Shrikanth S. Narayanan,et al.  Spoken dialog systems for children , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[5]  Diane J. Litman,et al.  Recognizing emotions from student speech in tutoring dialogues , 2003, 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721).

[6]  Frank Dellaert,et al.  Recognizing emotion in speech , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[7]  Shrikanth S. Narayanan,et al.  Politeness and frustration language in child-machine interactions , 2001, INTERSPEECH.

[8]  Shrikanth S. Narayanan,et al.  Toward detecting emotions in spoken dialogs , 2005, IEEE Transactions on Speech and Audio Processing.

[9]  Andreas Stolcke,et al.  Prosody-based automatic detection of annoyance and frustration in human-computer dialog , 2002, INTERSPEECH.