Listening Heads

In [1] we discussed functions of head movements and gaze. In this paper, we will go into more depth in the classification of various head movements: how they are distinguished in both formal and functional terms. We look at the distribution of a selection of primitive head movements and their related meanings and the way they are composed out of smaller units. This catalogue is not intended to be exhaustive and does not take into account the component items from other modalities such as speech, facial expressions, gestures and posture. The research is motivated by our desire to build systems that can interpret and mimick expressive human behaviour.

[1]  Jens Edlund,et al.  The Effect of Prosodic Features on the Interpretation of Synthesised Backchannels , 2006, PIT.

[2]  Peter D. Bricker,et al.  The role of audible and visible back-channel responses in interpersonal communication. , 1977 .

[3]  A. Manstead,et al.  Temporal Aspects of Facial Displays in Person and Expression Perception: The Effects of Smile Dynamics, Head-tilt, and Gender , 2007 .

[4]  Dirk Heylen,et al.  Searching for Prototypical Facial Feedback Signals , 2007, IVA.

[5]  Robert Hopper,et al.  Back Channels Revisited: Acknowledgment Tokens and Speakership Incipiency , 1993 .

[6]  S. Maynard.  Conversation management in contrast: Listener response in Japanese and American English , 1990 .

[7]  A. Kendon.  SOME FUNCTIONS OF THE FACE IN A KISSING ROUND , 1975 .

[8]  Ronald V. White,et al.  Back channelling, repair, pausing, and private speech , 1997 .

[9]  Hennie Brugman,et al.  Annotating Multi-media/Multi-modal Resources with ELAN , 2004, LREC.

[10]  T. Chartrand,et al.  The chameleon effect: the perception-behavior link and social interaction. , 1999, Journal of personality and social psychology.

[11]  Dirk Heylen,et al.  Computing Backchannel Distributions in Multi-Party Conversations , 2007, ACL 2007.

[12]  S. Duncan,et al.  On the structure of speaker–auditor interaction during speaking turns , 1974, Language in Society.

[13]  J. Atkinson,et al.  A change-of-state token and aspects of its sequential placement , 1985 .

[14]  C. Peterson,et al.  On the gender differential use of listener responsiveness , 1993 .

[15]  M. Stubbe.  Are you listening? Cultural influences on the use of supportive verbal feedback in conversation , 1998 .

[16]  Norihiro Hagita,et al.  Messages embedded in gaze of interface agents --- impression management with agent's gaze , 2002, CHI.

[17]  Jean Carletta,et al.  A shallow model of backchannel continuers in spoken dialogue , 2003 .

[18]  J. Gumperz.  Discourse strategies: Subject index , 1982 .

[19]  C. Goodwin.  Between and within: Alternative sequential treatments of continuers and assessments , 1986 .

[20]  Samuel S. Snyder,et al.  Young children's perception of nonverbally expressed “preference”: The effects of nonverbal cue, viewer age, and sex of actor , 1982 .

[21]  C. Gallois,et al.  The last ten turns: Behavior and sequencing in friends' and strangers' conversational findings , 1985 .

[22]  G. Jefferson.  Is “no” an acknowledgment token? Comparing American and British uses of (+)/(−) tokens , 2002 .

[23]  Starkey D. Duncan.  Language, Paralanguage, and Body Motion in the Structure of Conversations , 1975 .

[24]  Anne H. Anderson,et al.  The Hcrc Map Task Corpus , 1991 .

[25]  Stacy Marsella,et al.  Natural Behavior of a Listening Agent , 2005, IVA.

[26]  J. Bavelas,et al.  Listeners as co-narrators. , 2000, Journal of personality and social psychology.

[27]  L. J. Brunner,et al.  Smiles can be back channels. , 1979 .

[28]  Julia Hirschberg,et al.  The Prosody of Backchannels in American English , 2007 .

[29]  J. E. Tree.  Listeners' uses of um and uh in speech comprehension. , 2001 .

[30]  Nigel Ward,et al.  A prosodic feature that invites back-channels in Egyptian Arabic , 2007 .

[31]  Don H. Zimmerman,et al.  Acknowledgment Tokens and Speakership Incipiency Revisited , 1993 .

[32]  A. Dittmann,et al.  Relationship between vocalizations and head nods as listener responses. , 1968, Journal of personality and social psychology.

[33]  Sheida White.  Backchannels across cultures: A study of Americans and Japanese , 1989, Language in Society.

[34]  D. Fujimoto,et al.  Listener Responses in Interaction : A Case for Abandoning the Term, Backchannel , 2007 .

[35]  R. Rosenthal,et al.  Rapport expressed through nonverbal behavior , 1985 .

[36]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[37]  R. Hayashi,et al.  Floor structure of English and Japanese conversation , 1991 .

[38]  Steven H. Lewis,et al.  Listener Responsiveness and the Coordination of Conversation , 1982 .

[39]  Dirk Heylen,et al.  Facial Feedback Signals for ECAs , 2007 .

[40]  H. H. Clark,et al.  Speaking while monitoring addressees for understanding , 2004 .

[41]  V. Yngve.  On getting a word in edgewise , 1970 .

[42]  Loredana Cerrato.  Some characteristics of feedback expressions in Swedish , 2007 .

[43]  Tim Pitt,et al.  Physiology of Sport and Exercise , 2004 .

[44]  Alastair J. Gill,et al.  Coordination in Conversation and Rapport , 2007 .

[45]  Jens Edlund,et al.  User responses to prosodic variation in fragmentary grounding utterances in dialog , 2006, INTERSPEECH.

[46]  H. H. Clark,et al.  Hearers and speech acts , 1982 .

[47]  James S. Simkin.  Kinesics and Context: Essays on Body Motion Communication , 1972 .

[48]  A. Chaudhuri,et al.  The Many Faces of a Neutral Face: Head Tilt and Perception of Dominance and Emotion , 2003 .

[49]  A. Kendon.  Movement coordination in social interaction: some examples described. , 1970, Acta psychologica.

[50]  Julie Reid,et al.  A study of gender differences in minimal responses , 1995 .

[51]  D.K.J. Heylen,et al.  Multimodal Backchannel Generation for Conversational Agents , 2007 .

[52]  Adam Kendon,et al.  Some uses of the head shake , 2002 .

[53]  Evelyn Z. McClave.  Linguistic functions of head movements in the context of speech , 2000 .

[54]  Irving Adler,et al.  Language and man , 1970 .

[55]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[56]  S. Itahashi,et al.  Insertion of interjectory response based on prosodic information , 1996, Proceedings of IVTTA '96. Workshop on Interactive Voice Technology for Telecommunications Applications.

[57]  Uri Hadar,et al.  Kinematics of head movements accompanying speech during conversation , 1983 .

[58]  Herbert H. Clark,et al.  Contributing to Discourse , 1989, Cogn. Sci..

[59]  Katsuhiko Shirai,et al.  Analysis of head movements and its role in spoken dialogue , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[60]  Jean Carletta,et al.  The AMI Meeting Corpus: A Pre-announcement , 2005, MLMI.

[61]  Candace L. Sidner,et al.  Attentional Gestures in Dialogues Between People and Robots , 2007 .

[62]  Douglas A. Reynolds,et al.  An overview of automatic speaker diarization systems , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[63]  U. Hadar,et al.  Head movement during listening turns in conversation , 1985 .

[64]  Nicole Chovil.  Social determinants of facial displays , 1991 .

[65]  Rod Gardner,et al.  Between Speaking and Listening: The Vocalisation of Understandings1 , 1998 .

[66]  Seiichi Nakagawa,et al.  Timing Detection for Realtime Dialog Systems Using Prosodic and Linguistic Information , 2004 .

[67]  Klaus Krippendorff,et al.  Estimating the Reliability, Systematic Error and Random Error of Interval Data , 1970 .

[68]  Joakim Nivre,et al.  On the Semantics and Pragmatics of Linguistic Feedback , 1992, J. Semant..

[69]  I. Poggi,et al.  Perception of non-verbal emotional listener feedback , 2006 .

[70]  M. Argyle,et al.  The Different Functions of Gaze , 1973 .

[71]  Lynette Hirschman.  Female–male differences in conversational interaction , 1994 .

[72]  A. Lomax.  Cross cultural factors in phonological change , 1973, Language in Society.

[73]  Gary Bente,et al.  Person perception effects of computer-simulated male and female head movement , 1996 .

[74]  M. S. Feke,et al.  Effects of Native-Language and Sex on Back-Channel Behavior , 2003 .

[75]  Björn Granström,et al.  Multimodal feedback cues in human-machine interactions , 2002, Speech Prosody 2002.

[76]  J. Allwood,et al.  A study of gestural feedback expressions , 2006 .

[77]  Stefan Kopp,et al.  Modeling Embodied Feedback with Virtual Humans , 2006, ZiF Workshop.

[78]  E. Goffman,et al.  Forms of talk , 1982 .

[79]  Ning Wang,et al.  Creating Rapport with Virtual Agents , 2007, IVA.

[80]  J. Dixon,et al.  Gender, social context, and backchannel responses. , 1998, The Journal of social psychology.

[81]  Charles Goodwin,et al.  Concurrent Operations on Talk: Notes on the Interactive Organization of Assessments , 1987 .

[82]  C. Goodwin.  The Interactive Construction of a Sentence in Natural Conversation , 1979 .

[83]  A. Kendon.  Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[84]  Anna M. Guthrie,et al.  On the Systematic Deployment of Okay and Mmhmm in Academic Advising Sessions , 1997 .

[85]  K. Chang,et al.  Embodiment in conversational interfaces: Rea , 1999, CHI '99.

[86]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[87]  Yasuharu Den,et al.  Prosody-based detection of the context of backchannel responses , 1998, ICSLP.

[88]  Jeffery A. Jones,et al.  Visual Prosody and Speech Intelligibility , 2004, Psychological science.

[89]  Kristinn R. Thórisson,et al.  The Power of a Nod and a Glance: Envelope Vs. Emotional Feedback in Animated Conversational Agents , 1999, Appl. Artif. Intell..

[90]  Seiichi Nakagawa,et al.  A Spoken Dialog System for Chat-Like Conversations Considering Response Timing , 2007, TSD.

[91]  Philippe Blache,et al.  Backchannels revisited from a multimodal perspective , 2007, AVSP.

[92]  D. Horton,et al.  Mass communication and para-social interaction; observations on intimacy at a distance. , 1956, Psychiatry.

[93]  Jean E. Fox Tree,et al.  Listening in on monologues and dialogues , 1999 .

[94]  Dirk Heylen,et al.  Head Gestures, Gaze and the Principles of Conversational Structure , 2006, Int. J. Humanoid Robotics.

[95]  Susan Brennan,et al.  Interaction and feedback in a spoken language system: a theoretical framework , 1995, Knowl. Based Syst..

[96]  R. Birdwhistell.  Kinesics and context , 1970 .

[97]  Campbell Leaper,et al.  Self-disclosure and listener verbal support in same-gender and cross-gender friends' conversations , 1995 .

[98]  Boris Reuderink,et al.  The Influence of Gaze and Head Tilt on the Impression of Listening Agents , 2006 .

[99]  Marco Costa,et al.  Head Canting in Paintings: An Historical Study , 2001 .

[100]  Loredana Cerrato,et al.  Analysis and measurement of head movements signalling feedback in face-to-face human dialogue , 2003 .

[101]  S. W. Gregory,et al.  Conversation partner mutual adaptation as demonstrated by Fourier series analysis , 1982 .

[102]  A. Ichikawa,et al.  An Analysis of Turn-Taking and Backchannels Based on Prosodic and Syntactic Features in Japanese Map Task Dialogs , 1998, Language and speech.

[103]  Stacy Marsella,et al.  Virtual Rapport , 2006, IVA.

[104]  Anna M. Fellegy.  Patterns and functions of minimal response , 1995 .

[105]  Dirk Heylen,et al.  Experimenting with the Gaze of a Conversational Agent , 2002 .

[106]  Marco Costa,et al.  Social Presence, Embarrassment, and Nonverbal Behavior , 2001 .