The analysis of embodied communicative feedback in multimodal corpora: a prerequisite for behavior simulation

Communicative feedback refers to unobtrusive (usually short) vocal or bodily expressions whereby a recipient of information can inform a contributor of information about whether he/she is able and willing to communicate, perceive the information, and understand the information. This paper provides a theory for embodied communicative feedback, describing the different dimensions and features involved. It also provides a corpus analysis part, describing a first data coding and analysis method geared to find the features postulated by the theory. The corpus analysis part describes different methods and statistical procedures and discusses their applicability and the possible insights gained with these methods.

[1]  E. Robinson.  Cybernetics, or Control and Communication in the Animal and the Machine , 1963 .

[2]  Costanza Navarretta,et al.  The MUMIN multimodal coding scheme , 2005 .

[3]  C. Hartshorne,et al.  Collected Papers of Charles Sanders Peirce , 1935, Nature.

[4]  Roger Bakeman,et al.  Analyzing Interaction: Sequential Analysis with SDIS and GSEQ , 1995 .

[5]  Nigel G. Ward,et al.  Prosodic features which cue back-channel responses in English and Japanese , 2000 .

[6]  Karl R. Popper.  The Logic of Scientific Discovery. , 1977 .

[7]  M. Martin Taylor,et al.  The Structure of Multimodal Dialogue II , 2001, Computational Linguistics.

[8]  K. Grammer,et al.  Fuzziness of nonverbal courtship communication unblurred by motion energy detection. , 1999, Journal of personality and social psychology.

[9]  C. Peirce,et al.  Collected Papers of Charles Sanders Peirce , 1936, Nature.

[10]  K. Popper,et al.  Logik der Forschung , 1935 .

[11]  M S Magnusson,et al.  Discovering hidden time patterns in behavior: T-patterns and their detection , 2000, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[12]  M. Magnússon.  Hidden Real-Time Patterns in Intra- and Inter-Individual Behavior: Description and Detection , 1996 .

[13]  J. Allwood.  Linguistic communication as action and cooperation : a study in pragmatics , 1976 .

[14]  Jens Allwood,et al.  The structure of dialog , 1999 .

[15]  K. Grammer,et al.  Non-verbal behavior as courtship signals: the role of control and choice in selecting partners. , 2000, Evolution and human behavior : official journal of the Human Behavior and Evolution Society.

[16]  Joakim Nivre,et al.  On the Semantics and Pragmatics of Linguistic Feedback , 1992, J. Semant..

[17]  Stefan Kopp,et al.  A Conversational Agent as Museum Guide - Design and Evaluation of a Real-World Application , 2005, IVA.

[18]  Stefan Kopp,et al.  Modeling Embodied Feedback with Virtual Humans , 2006, ZiF Workshop.