Using Probabilistic Feature Matching to Understand Spoken Descriptions

We describe a probabilistic reference disambiguation mechanism developed for a spoken dialogue system mounted on an autonomous robotic agent. Our mechanism performs probabilistic comparisons between features specified in referring expressions (e.g. size and colour) and features of objects in the domain. The results of these comparisons are combined using a function weighted on the basis of the specified features. Our evaluation shows high reference resolution accuracy across a range of spoken referring expressions.

[1]  Christiane Fellbaum,et al.  Combining Local Context and Wordnet Similarity for Word Sense Identification , 1998 .

[2]  Andreas Stolcke,et al.  Prosody-based automatic detection of annoyance and frustration in human-computer dialog , 2002, INTERSPEECH.

[3]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[4]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[5]  Advaith Siddharthan,et al.  Generating Referring Expressions in Open Domains , 2004, ACL.

[6]  John D. Kelleher.  Attention driven reference resolution in multimodal contexts , 2006, Artificial Intelligence Review.

[7]  Jean Carletta,et al.  Assessing Agreement on Classification Tasks: The Kappa Statistic , 1996, CL.

[8]  John F. Sowa,et al.  Conceptual Structures: Information Processing in Mind and Machine , 1983 .

[9]  Ingrid Zukerman,et al.  A Probabilistic Approach to the Interpretation of Spoken Utterances , 2008, PRICAI.

[10]  Jeremy L Wyatt.  Planning clarification questions to resolve ambiguous references to objects , 2005 .

[11]  Chalapathy Neti,et al.  Stream confidence estimation for audio-visual speech recognition , 2000, INTERSPEECH.

[12]  Joachim M. Buhmann,et al.  Empirical evaluation of dissimilarity measures for color and texture , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Yasushi Makihara,et al.  Object recognition supported by user interaction for service robots , 2002, Object recognition supported by user interaction for service robots.

[14]  Ted Pedersen,et al.  WordNet::Similarity - Measuring the Relatedness of Concepts , 2004, NAACL.

[15]  Jan Alexandersson,et al.  A Robust and Generic Discourse Model for Multimodal Dialogue , 2003 .