Incorporating a user model to improve detection of unhelpful robot answers

Dialogues with robots frequently exhibit social dialogue acts such as greeting, thanks, and goodbye. This opens the opportunity of using these dialogue acts for dialogue management, in particular for detecting misunderstandings. Our corpus analysis shows that the social dialogue acts have different scopes of their associations with the discourse features within the dialogue: greeting in the user's first turn is associated with such distant, or global, features as the likelihood of having questions answered, persistence, and ending with bye. The user's thanks turn, on the other hand, is strongly associated with the helpfulness of the preceding robot's answer. We therefore interpret the greeting as a component of a user model that can provide information about the user's traits and be associated with discourse features at various stages of the dialogue. We conduct a detailed analysis of the user's thanking behavior and demonstrate that user's thanks can be used in the detection of unhelpful robot's answers. Incorporating the greeting information further improves the detection. We discuss possible applications of this work for human-robot dialogue management.

[1]  Emiel Krahmer,et al.  Problem spotting in human-machine interaction , 1999, EUROSPEECH.

[2]  Jens Edlund,et al.  Early error detection on word level , 2004 .

[3]  David Maxwell Chickering,et al.  Evaluating the Markov assumption in Markov Decision Processes for spoken dialogue management , 2006, Lang. Resour. Evaluation.

[4]  Kerstin Fischer,et al.  Keeping the Initiative: An Empirically-Motivated Approach to Predicting User-Initiated Dialogue Contribution in HCI , 2006, EACL.

[5]  Mikio Nakano,et al.  New feature parameters for detecting misunderstandings in a spoken dialogue system , 2000, INTERSPEECH.

[6]  Herbert H. Clark,et al.  Contributing to Discourse , 1989, Cogn. Sci..

[7]  Emiel Krahmer,et al.  Error Detection in Spoken Human-Machine Interaction , 2001, Int. J. Speech Technol..

[8]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[9]  Maxine Eskénazi,et al.  Optimizing Endpointing Thresholds using Dialogue Features in a Spoken Dialogue System , 2008, SIGDIAL Workshop.

[10]  Julia Hirschberg,et al.  Predicting Automatic Speech Recognition Performance Using Prosodic Cues , 2000, ANLP.

[11]  Massimo Poesio,et al.  The predictive power of game structure in dialogue act recognition: experimental results using maximum entropy estimation , 1998, ICSLP.

[12]  Pat Langley,et al.  A Personalized System for Conversational Recommendations , 2011, J. Artif. Intell. Res..

[13]  Min Kyung Lee,et al.  How do people talk with a robot?: an analysis of human-robot dialogues in the real world , 2009, CHI Extended Abstracts.

[14]  Julia Hirschberg,et al.  Corrections in spoken dialogue systems , 2000, INTERSPEECH.

[15]  Alan M. Lesgold,et al.  Modeling the process of collaborative learning , 2007 .

[16]  Diane J. Litman,et al.  Adapting to Student Uncertainty Improves Tutoring Dialogues , 2009, AIED.

[17]  Norbert Reithinger,et al.  Dialogue act classification using language models , 1997, EUROSPEECH.

[18]  Kerstin Fischer,et al.  The Role of Users ’ Preconceptions in Talking to Computers and Robots , 2006 .

[19]  Guido Zarrella,et al.  Using Dialogue Features to Predict Trouble During Collaborative Learning , 2005, User Modeling and User-Adapted Interaction.

[20]  Diane J. Litman,et al.  Designing and evaluating a wizarded uncertainty-adaptive spoken dialogue tutoring system , 2011, Comput. Speech Lang..

[21]  Reid G. Simmons,et al.  Relating initial turns of human-robot dialogues to discourse , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[22]  Sofus A. Macskassy,et al.  Confidence Bands for Roc Curves , 2003 .

[23]  Claire Grover,et al.  In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC , 2006 .

[24]  Stephanie Rosenthal,et al.  Designing robots for long-term social interaction , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Harry Bunt,et al.  Dimensions in Dialogue Act Annotation , 2006, LREC.