Multimodal Analysis of Laughter for an Interactive System

In this paper, we focus on the development of new methods to detect and analyze laughter, in order to enhance human-computer interactions. First, the general architecture of such a laughter-enabled application is presented. Then, we propose the use of two new modalities, namely body movements and respiration, to enrich the audiovisual laughter detection and classification phase. These additional signals are acquired using easily constructed affordable sensors. Features to characterize laughter from body movements are proposed, as well as a method to detect laughter from a measure of thoracic circumference.

[1]  Willibald Ruch,et al.  Exhilaration and humor. , 1993 .

[2]  David A. van Leeuwen,et al.  Automatic discrimination between laughter and speech , 2007, Speech Commun..

[3]  Nikki Mirghafori,et al.  Automatic laughter detection using neural networks , 2007, INTERSPEECH.

[4]  Radoslaw Niewiadomski,et al.  Laugh-aware virtual agent and its impact on user amusement , 2013, AAMAS.

[5]  Hiroyuki Kajimoto,et al.  Laugh enhancer using laugh track synchronized with the user's laugh motion , 2010, CHI Extended Abstracts.

[6]  Maurizio Mancini,et al.  Computing and Evaluating the Body Laughter Index , 2012, HBU.

[7]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[8]  Radoslaw Niewiadomski,et al.  Visual laughter synthesis: Initial approaches , 2012 .

[9]  Dietmar Todt,et al.  Laughter in Conversation: Features of Occurrence and Acoustic Structure , 2004 .

[10]  Thierry Dutoit,et al.  A Phonetic Analysis of Natural Laughter, for Use in Automatic Laughter Processing Systems , 2011, ACII.

[11]  P. Ekman,et al.  The expressive pattern of laughter , 2001 .

[12]  Antonio Camurri,et al.  Developing multimodal interactive systems with EyesWeb XMI , 2007, NIME '07.

[13]  Radoslaw Niewiadomski,et al.  Cross-media agent platform , 2011, Web3D '11.

[14]  Daniel Vélez Día,et al.  Biomechanics and Motor Control of Human Movement , 2013 .

[15]  Daniel P. W. Ellis,et al.  Laughter Detection in Meetings , 2004 .

[16]  I Iandelli,et al.  Respiratory dynamics during laughter. , 2001, Journal of applied physiology.

[17]  C. Pelachaud,et al.  Enabling a virtual agent to join in laughing with a conversational partner using a similarity-driven audiovisual laughter animation , 2010 .

[18]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[19]  Hatice Gunes,et al.  Human Behavior Understanding , 2016, Lecture Notes in Computer Science.

[20]  Maja Pantic,et al.  Fusion of audio and visual cues for laughter detection , 2008, CIVR '08.

[21]  William A. Sethares,et al.  Periodicity transforms , 1999, IEEE Trans. Signal Process..

[22]  Antoinette M. Feleky.  The influence of the emotions on respiration. , 1916 .

[23]  Maja Pantic,et al.  Audiovisual Discrimination Between Speech and Laughter: Why and When Visual Information Might Help , 2011, IEEE Transactions on Multimedia.