Prosodic Cues to Disengagement and Uncertainty in Physics Tutorial Dialogues

This paper focuses on the analysis and prediction of student disengagement and uncertainty, using a corpus of dialogues collected with a spoken tutorial dialogue system in the STEM domain of qualitative physics. We first compare and contrast the prosodic characteristics of dialogue turns exhibiting disengagement or not, and those exhibiting uncertainty or not. We then compare the utility of using multiple prosodic features to predict both disengagement and uncertainty.

[1]  Björn Schuller,et al.  Being bored? Recognising natural interest by extensive audiovisual integration for real-life application , 2009, Image Vis. Comput..

[2]  Björn Schuller,et al.  Opensmile: the munich versatile and fast open-source audio feature extractor , 2010, ACM Multimedia.

[3]  Andreas Stolcke,et al.  Prosody-based automatic detection of annoyance and frustration in human-computer dialog , 2002, INTERSPEECH.

[4]  Björn W. Schuller,et al.  The INTERSPEECH 2010 paralinguistic challenge , 2010, INTERSPEECH.

[5]  Tim Paek,et al.  Accommodating explicit user expressions of uncertainty in voice search or something like that , 2008, INTERSPEECH.

[6]  Stuart M. Shieber,et al.  Recognizing Uncertainty in Speech , 2011, EURASIP J. Adv. Signal Process..

[7]  Arthur C. Graesser,et al.  A Time for Emoting: When Affect-Sensitivity Is and Isn't Effective at Promoting Deep Learning , 2010, Intelligent Tutoring Systems.

[8]  Joanna Drummond,et al.  Intrinsic and Extrinsic Evaluation of an Automatic User Disengagement Detector for an Uncertainty-Adaptive Spoken Dialogue System , 2012, NAACL.

[9]  Brady Clark,et al.  Responding to Student Uncertainty in Spoken Tutorial Dialogue Systems , 2006, Int. J. Artif. Intell. Educ..

[10]  Diane J. Litman,et al.  Benefits and challenges of real-time uncertainty detection and adaptation in a spoken dialogue computer tutor , 2011, Speech Commun..

[11]  Jack Mostow,et al.  A Tale of Two Tasks: Detecting Children's Off-Task Speech in a Reading Tutor , 2011, INTERSPEECH.