Are You Being Addressed? - Real-Time Addressee Detection to Support Remote Participants in Hybrid Meetings

In this paper, we describe the development of a meeting assistant agent that helps remote meeting participants by notifying them when they are being addressed. We present experiments that have been conducted to develop machine classifiers to decide whether "you are being addressed" where "you" refers to a fixed (remote) participant in a meeting. The experimental results back up the choices made regarding the selection of data, features, and classification methods. We discuss variations of the addressee classification problem that have been considered in the literature and how suitable they are for addressee detection in a system that plays a role in a live meeting.

[1]  Olga Kulyk,et al.  Engagement and Floor Control in Hybrid Meetings , 2010 .

[2]  Jacques M. B. Terken,et al.  Facial Orientation During Multi-party Interaction with Information Kiosks , 2003, INTERACT.

[3]  Carole Edelsky.  Who's got the floor? , 1981, Language in Society.

[4]  Shinji Ozawa,et al.  Automatic Addressee Identification Based on Participants' Head Orientation and Utterances for Multiparty Conversations , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[5]  Alistair Knott,et al.  Multi-agent Human-Machine Dialogue: Issues in Dialogue Management and Referring Expression Semantics , 2004, PRICAI.

[6]  Stanley Peters,et al.  Who is “You”? Combining Linguistic and Gaze Features to Resolve Second-Person References in Dialogue , 2009, EACL.

[7]  Mariët Theune,et al.  How Do I Address You? Modelling addressing behavior based on an analysis of a multi-modal corpora of conversational discourse , 2008 .

[8]  Alistair Knott,et al.  Multi-agent human-machine dialogue: issues in dialogue management and referring expression semantics , 2004, Artif. Intell..

[9]  Dennis Reidsma,et al.  Annotations and subjective machines of annotators, embodied agents, users, and other humans , 2008 .

[10]  Gene H. Lerner,et al.  On the place of linguistic resources in the organization of talk-in-interaction: A co-investigation of English and Japanese grammatical practices , 1999 .

[11]  Anton Nijholt,et al.  Addressee Identification in Face-to-Face Meetings , 2006, EACL.

[12]  Tanja Schultz,et al.  Identifying the addressee in human-human-robot interactions based on head pose and speech , 2004, ICMI '04.

[13]  Randall W. Hill,et al.  Steve Goes to Bosnia: Towards a New Generation of Virtual Humans for Interactive Experiences , 2001 .

[14]  Jean Carletta,et al.  Unleashing the killer corpus: experiences in creating the multi-everything AMI Meeting Corpus , 2007, Lang. Resour. Evaluation.

[15]  Jean Carletta,et al.  The AMI meeting corpus , 2005 .

[16]  Walter Daelemans,et al.  Combined Optimization of Feature Selection and Algorithm Parameter Interaction in Machine Learning of Language , 2003 .

[17]  Ian Witten,et al.  Data Mining , 2000 .

[18]  Rieks op den Akker,et al.  A comparison of addressee detection methods for multiparty conversations , 2009 .

[19]  David R. Traum,et al.  Issues in Multiparty Dialogues , 2003, Workshop on Agent Communication Languages.

[20]  Anton Nijholt,et al.  A corpus for studying addressing behaviour in multi-party dialogues , 2006, SIGDIAL.

[21]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[22]  Rieks op den Akker,et al.  Towards Automatic Addressee Identification in Multi-party Dialogues , 2004, SIGDIAL Workshop.

[23]  Randall W. Hill,et al.  Toward a New Generation of Virtual Humans for Interactive Experiences , 2002, IEEE Intell. Syst..

[24]  Thomas Hain,et al.  Automatic speech recognition for scientific purposes - webASR , 2008, INTERSPEECH.

[25]  Gene H. Lerner.  On the Place of Linguistic Resources in the Organization of Talk-in-Interaction: 'Second Person' Reference in Multi-Party Conversation , 1996 .

[26]  Jean Carletta,et al.  Squibs: Reliability Measurement without Limits , 2008, CL.

[27]  Berry Eggen,et al.  Identifying the intended addressee in mixed human-human and human-computer interaction from non-verbal features , 2005, ICMI '05.

[28]  Natasa Jovanovic,et al.  To whom it may concern : adressee identification in face-to-face meetings , 2007 .

[29]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[30]  Jean-Marc Odobez,et al.  Recognizing Visual Focus of Attention From Head Pose in Natural Meetings , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[31]  Surabhi Gupta,et al.  Resolving "You" in Multi-Party Dialog , 2007, SIGdial.

[32]  Stanley Peters,et al.  Detecting and Summarizing Action Items in Multi-Party Dialogue , 2007, SIGDIAL.

[33]  Wilfried Post,et al.  A research environment for meeting behavior , 2004 .