Towards an Annotation Scheme for Complex Laughter in Speech Corpora

Although laughter research has gained quite some interest over the past few years, a shared description of how to annotate laughter and its sub-units is still missing. We present a first attempt towards an annotation scheme that contributes to improving the homogeneity and transparency with which laughter is annotated. This includes the integration of respiratory noises as well as stretches of speech-laughs, and to a limited extend to smiled speech and short silent intervals. Inter-annotator agreement is assessed while applying the scheme to different corpora where laughter is evoked through different methods and varying settings. Annotating laughter becomes more complex when the situation in which laughter occurs becomes more spontaneous and social. There is a substantial disagreement among the annotators with respect to temporal alignment (when does a unit start and when does it end) and unit classification, particularly the determination of starts/ends of laughter episodes. In summary, this detailed laughter annotation study reflects the need for better investigations of the various components of laughter.

[1]  V. Tartter Happy talk: Perceptual and acoustic effects of smiling on speech , 1980, Perception & psychophysics.

[2]  D. Mowrer,et al.  Analysis of five acoustic correlates of laughter , 1987 .

[3]  E. Schegloff,et al.  Notes on Laughter in the Pursuit of Intimacy , 1987 .

[4]  Phillip J. Glenn Initiating shared laughter in multi‐party conversations , 1989 .

[5]  Sheri Hunnicutt,et al.  Acoustic analysis of laughter , 1992, ICSLP.

[6]  E. Nwokah,et al.  Vocal affect in three-year-olds: a quantitative acoustic analysis of child laughter. , 1993, The Journal of the Acoustical Society of America.

[7]  D. Mowrer A case study of perceptual and acoustic features of an infant’s first laugh utterances , 1994 .

[8]  E. Yanagisawa,et al.  Videoendoscopic Analysis of Laryngeal Function during Laughter , 1996, The Annals of otology, rhinology, and laryngology.

[9]  THE RITUALISATION OF LAUGHTER Karl Grammer Irenaus Eibl-Eibesfeldt CONTENTS 1. THE RITUALISATION OF BEHAVIOUR 2. METHODOLOGICAL ISSUES 3. DETERMINANTS OF RITUALISATION OF LAUGHTER 4. THE FUNCTION OF LAUGHTER BIBLIOGRAPHY 1. THE RITUALISATION OF BEHAVIOUR , 1998 .

[10]  A. Fogel,et al.  The integration of laughter and speech in vocal communication: a dynamic systems perspective. , 1999, Journal of speech, language, and hearing research : JSLHR.

[11]  J. Trouvain Phonetic Aspects of "Speech-Laughs" , 2001 .

[12]  J. Bachorowski,et al.  The acoustic features of human laughter. , 2001, The Journal of the Acoustical Society of America.

[13]  P. Ekman,et al.  The expressive pattern of laughter , 2001 .

[14]  Paul Boersma,et al.  Praat: doing phonetics by computer , 2003 .

[15]  J. Trouvain Segmenting Phonetic Units in Laughter , 2003 .

[16]  Andreas Stolcke,et al.  The ICSI Meeting Corpus , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[17]  Ralph Weischedel,et al.  PERFORMANCE MEASURES FOR INFORMATION EXTRACTION , 2007 .

[18]  David A. van Leeuwen,et al.  Automatic discrimination between laughter and speech , 2007, Speech Commun..

[19]  F. Gosselin,et al.  The Montreal Affective Voices: A validated set of nonverbal affect bursts for research on auditory affective processing , 2008, Behavior research methods.

[20]  R. Provine Laughter Punctuates Speech: Linguistic, Social and Gender Contexts of Laughter , 2010 .

[21]  E. Holt The last laugh: Shared laughter and topic termination , 2010 .

[22]  Thierry Dutoit,et al.  The AVLaughterCycle Database , 2010, LREC.

[23]  Thierry Dutoit,et al.  A Phonetic Analysis of Natural Laughter, for Use in Automatic Laughter Processing Systems , 2011, ACII.

[24]  Wallace L. Chafe,et al.  The Importance of Not Being Earnest – The Feeling behind Laughter and Humor , 2011, Phonetica.

[25]  Jürgen Trouvain,et al.  On the acoustics of overlapping laughter in conversational speech , 2012, INTERSPEECH.

[26]  Jürgen Trouvain,et al.  Laughter annotations in conversational speech corpora - possibilities and limitations for phonetic analysis , 2012 .

[27]  Maja Pantic,et al.  Audiovisual Detection of Laughter in Human-Machine Interaction , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[28]  Antje Schweitzer,et al.  Convergence of articulation rate in spontaneous speech , 2013, INTERSPEECH.

[29]  Thierry Dutoit,et al.  Arousal-Driven Synthesis of Laughter , 2014, IEEE Journal of Selected Topics in Signal Processing.

[30]  Yann Mathet,et al.  The Unified and Holistic Method Gamma (γ) for Inter-Annotator Agreement Measure and Alignment , 2015, CL.

[31]  Ye Tian,et al.  When do we laugh? , 2016, SIGDIAL Conference.

[32]  H. Rothgänger,et al.  Analysis of Laughter and Speech Sounds in Italian and German Students , 1998, The Science of Nature.

[33]  Deniece S. Nazareth,et al.  The MULAI Corpus: Multimodal Recordings of Spontaneous Laughter in Dyadic Interaction , 2018 .