Laugh machine

The Laugh Machine project aims at endowing virtual agents with the capability to laugh naturally, at the right moment and with the correct intensity, when interacting with human participants. In this report we present the technical development and evaluation of such an agent in one specific scenario: watching TV along with a participant. The agent must be able to react to both, the video and the participant’s behaviour. A full processing chain has been implemented, integrating components to sense the human behaviours, decide when and how to laugh and, finally, synthesize audiovisual laughter animations. The system was evaluated in its capability to enhance the affective experience of naive participants, with the help of pre and post-experiment questionnaires. Three interaction conditions have been compared: laughter-enabled or not, reacting to the participant’s behaviour or not. Preliminary results (the number of experiments is currently to small to obtain statistically significant differences) show that the interactive, laughter-enabled agent is positively perceived and is increasing the emotional dimension of the experiment.

[1]  G. S. Hall,et al.  The Psychology of Tickling, Laughing, and the Comic , 1897 .

[2]  Nikki Mirghafori,et al.  Automatic laughter detection using neural networks , 2007, INTERSPEECH.

[3]  P. Ekman,et al.  The expressive pattern of laughter , 2001 .

[4]  Antoinette M. Feleky.  The influence of the emotions on respiration. , 1916 .

[5]  吉村 貴克,et al.  Simultaneous modeling of phonetic and prosodic parameters,and characteristic conversion for HMM-based text-to-speech systems , 2002 .

[6]  H. Zen,et al.  An HMM-based speech synthesis system applied to English , 2002, Proceedings of 2002 IEEE Workshop on Speech Synthesis, 2002..

[7]  Hiroyuki Kajimoto,et al.  Laugh enhancer using laugh track synchronized with the user's laugh motion , 2010, CHI Extended Abstracts.

[8]  Hideki Kawahara,et al.  STRAIGHT, exploitation of the other aspect of VOCODER: Perceptually isomorphic decomposition of speech sounds , 2006 .

[9]  Willibald Ruch,et al.  To be in good or bad humour: Construction of the state form of the State-Trait-Cheerfulness-inventory—STCI , 1997 .

[10]  Björn Schuller,et al.  Opensmile: the munich versatile and fast open-source audio feature extractor , 2010, ACM Multimedia.

[11]  C. Pelachaud,et al.  Enabling a virtual agent to join in laughing with a conversational partner using a similarity-driven audiovisual laughter animation , 2010 .

[12]  H. Ishiguro,et al.  Laughter in Social Robotics – no laughing matter , 2009 .

[13]  Leonid Sigal,et al.  Facial Expression Transfer with Input-Output Temporal Restricted Boltzmann Machines , 2011, NIPS.

[14]  Willibald Ruch,et al.  A temperament approach to humor , 2007 .

[15]  Willibald Ruch,et al.  Extending the study of gelotophobia: On gelotophiles and katagelasticists , 2009 .

[16]  W. Ruch,et al.  Assessing the „humorous temperament“: Construction of the facet and standard trait forms of the State-Trait-Cheerfulness-Inventory — STCI , 1996 .

[17]  David A. van Leeuwen,et al.  Automatic discrimination between laughter and speech , 2007, Speech Commun..

[18]  Thierry Dutoit,et al.  The Deterministic Plus Stochastic Model of the Residual Signal and Its Applications , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Willibald Ruch,et al.  The fear of being laughed at: Individual and group differences in Gelotophobia , 2008 .

[20]  Victor B. Zordan,et al.  Laughing out loud: control for modeling anatomically inspired laughter using audio , 2008, SIGGRAPH 2008.

[21]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[22]  I Iandelli,et al.  Respiratory dynamics during laughter. , 2001, Journal of applied physiology.

[23]  Daniel Vélez Día,et al.  Biomechanics and Motor Control of Human Movement , 2013 .

[24]  Antonio Camurri,et al.  Developing multimodal interactive systems with EyesWeb XMI , 2007, NIME '07.

[25]  Thierry Dutoit,et al.  Finding out the audio and visual features that influence the perception of laughter intensity and differ in inhalation and exhalation phases , 2012 .

[26]  Hiroshi Ishiguro,et al.  How about laughter? Perceived naturalness of two laughing humanoid robots , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[27]  Johannes Wagner,et al.  The Social Signal Interpretation Framework (SSI) for Real Time Signal Processing and Recognition , 2011, INTERSPEECH.

[28]  Thierry Dutoit,et al.  A Phonetic Analysis of Natural Laughter, for Use in Automatic Laughter Processing Systems , 2011, ACII.

[29]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[30]  Alfred W. Kaszniak,et al.  Emotions, Qualia, and Consciousness , 2001 .

[31]  J. Trouvain,et al.  IMITATING CONVERSATIONAL LAUGHTER WITH AN ARTICULATORY SPEECH SYNTHESIZER , 2007 .

[32]  Daniel P. W. Ellis,et al.  Laughter Detection in Meetings , 2004 .

[33]  Maja Pantic,et al.  The SEMAINE corpus of emotionally coloured character interactions , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[34]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[35]  Maja Pantic,et al.  Audiovisual laughter detection based on temporal features , 2008, ICMI '08.

[36]  Maja Pantic,et al.  Classifying laughter and speech using audio-visual feature prediction , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[37]  Shrikanth Narayanan,et al.  Automatic acoustic synthesis of human-like laughter. , 2007, The Journal of the Acoustical Society of America.

[38]  Thierry Dutoit,et al.  The AVLaughterCycle Database , 2010, LREC.

[39]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[40]  B. N. Barman.  Laughing , Crying , Sneezing and Yawning : Automatic Voice Dr iven Animation of Non-Speech Articulations ∗ , 2006 .

[41]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[42]  Maja Pantic,et al.  Fusion of audio and visual cues for laughter detection , 2008, CIVR '08.

[43]  William A. Sethares,et al.  Periodicity transforms , 1999, IEEE Trans. Signal Process..

[44]  Maja Pantic,et al.  Audiovisual discrimination between laughter and speech , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  Emiel Krahmer,et al.  Exploring social and temporal dimensions of emotion induction using an adaptive affective mirror , 2009, CHI Extended Abstracts.