DeHumor: Visual Analytics for Decomposing Humor

Despite being a critical communication skill, grasping humor is challenginga successful use of humor requires a mixture of both engaging content build-up and an appropriate vocal delivery (e.g., pause). Prior studies on computational humor emphasize the textual and audio features immediately next to the punchline, yet overlooking longer-term context setup. Moreover, the theories are usually too abstract for understanding each concrete humor snippet. To fill in the gap, we develop DeHumor, a visual analytical system for analyzing humorous behaviors in public speaking. To intuitively reveal the building blocks of each concrete example, DeHumor decomposes each humorous video into multimodal features and provides inline annotations of them on the video script. In particular, to better capture the build-ups, we introduce content repetition as a complement to features introduced in theories of computational humor and visualize them in a context linking graph. To help users locate the punchlines that have the desired features to learn, we summarize the content (with keywords) and humor feature statistics on an augmented time matrix. With case studies on stand-up comedy shows and TED talks, we show that DeHumor is able to highlight various building blocks of humor examples. In addition, expert interviews with communication coaches and humor researchers demonstrate the effectiveness of DeHumor for multimodal humor analysis of speech content and vocal delivery.

[1]  J. Mulholland,et al.  A Handbook of Persuasive Tactics: A Practical Language Guide , 1993 .

[2]  R. Bauman,et al.  Story, Performance, and Event: Contextual Studies of Oral Narrative , 1986 .

[3]  Carlo Strapparava,et al.  LEARNING TO LAUGH (AUTOMATICALLY): COMPUTATIONAL MODELS FOR HUMOR RECOGNITION , 2006, Comput. Intell..

[4]  Anna Rumshisky,et al.  HumorHawk at SemEval-2017 Task 6: Mixing Meaning and Sound for Humor Recognition , 2017, SemEval@ACL.

[5]  Davide Buscaldi,et al.  From humor recognition to irony detection: The figurative language of social media , 2012, Data Knowl. Eng..

[6]  Rada Mihalcea,et al.  The language of humour , 2010 .

[7]  Björn W. Schuller,et al.  The INTERSPEECH 2010 paralinguistic challenge , 2010, INTERSPEECH.

[8]  Catherine Plaisant,et al.  What's being said near “Martha”? Exploring name entities in literary text collections , 2009, 2009 IEEE Symposium on Visual Analytics Science and Technology.

[9]  P. Wooten,et al.  Humor: an antidote for stress. , 1996, Holistic nursing practice.

[10]  Mireia Farrús,et al.  Prosograph: A Tool for Prosody Visualisation of Large Speech Corpora , 2017, INTERSPEECH.

[11]  Diane J. Litman,et al.  Humor: Prosody Analysis and Automatic Recognition for F*R*I*E*N*D*S* , 2006, EMNLP.

[12]  Rada Mihalcea,et al.  Characterizing Humour: An Exploration of Features in Humorous Texts , 2009, CICLing.

[13]  Pascale Fung,et al.  Multimodal deep neural nets for detecting humor in TV sitcoms , 2016, 2016 IEEE Spoken Language Technology Workshop (SLT).

[14]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[15]  J. P. Davidson The Complete Guide to Public Speaking , 2002 .

[16]  DebateVis: Visualizing Political Debates for Non-Expert Users , 2020, 2020 IEEE Visualization Conference (VIS).

[17]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[18]  S. Attardo,et al.  Multimodality in conversational humor , 2013 .

[19]  Akira Watanabe,et al.  Speech visualization by integrating features for the hearing impaired , 2000, IEEE Trans. Speech Audio Process..

[20]  Maja Pantic,et al.  Is this joke really funny? judging the mirth by audiovisual laughter analysis , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[21]  J. Kemp,et al.  The World As Will And Idea , 1975 .

[22]  Victor Raskin,et al.  Semantic mechanisms of humor , 1984 .

[23]  Navjyoti Singh,et al.  What makes us laugh? Investigations into Automatic Humor Classification , 2018, PEOPLES@NAACL-HTL.

[24]  Marco Büchler,et al.  Visualizations for Text Re-use , 2015, 2014 International Conference on Information Visualization Theory and Applications (IVAPP).

[25]  Guillermo Moncecchi,et al.  A Crowd-Annotated Spanish Corpus for Humor Analysis , 2017, SocialNLP@ACL.

[26]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[27]  Maja Pantic,et al.  The MAHNOB Laughter database , 2013, Image Vis. Comput..

[28]  J. Morreall The Philosophy of laughter and humor , 1987 .

[29]  Julia M. Taylor,et al.  Computational Detection of Humor: A Dream or a Nightmare? The Ontological Semantics Approach , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[30]  Jeannett Martin,et al.  English Text: System and structure , 1992 .

[31]  Iryna Gurevych,et al.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , 2019, EMNLP.

[32]  Benno Stein,et al.  Visual Assessment of Alleged Plagiarism Cases , 2015, Comput. Graph. Forum.

[33]  Xiaojuan Ma,et al.  Recognizing Humour using Word Associations and Humour Anchor Extraction , 2018, COLING.

[34]  John Sinclair,et al.  Corpus, Concordance, Collocation , 1991 .

[35]  Julia Taylor Rayz,et al.  Computationally Recognizing Wordplay in Jokes , 2004 .

[36]  Diyi Yang,et al.  Humor Recognition and Humor Anchor Extraction , 2015, EMNLP.

[37]  S. Attardo,et al.  Timing in the performance of jokes , 2011 .

[38]  D. Tannen Talking Voices: Repetition, Dialogue, and Imagery in Conversational Discourse , 1989 .

[39]  Dragomir R. Radev,et al.  Humor in Collective Discourse: Unsupervised Funniness Detection in the New Yorker Cartoon Caption Contest , 2015, LREC.

[40]  Yuriy Brun,et al.  That's What She Said: Double Entendre Identification , 2011, ACL.

[41]  Gautham J. Mysore,et al.  Capture-Time Feedback for Recording Scripted Narration , 2015, UIST.

[42]  Ted Pedersen,et al.  Duluth at SemEval-2017 Task 6: Language Models in Humor Detection , 2017, SemEval@ACL.

[43]  Bettina Berendt,et al.  Web Mining for Understanding Stories through Graph Visualisation , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[44]  Martin Wattenberg,et al.  Arc diagrams: visualizing structure in strings , 2002, IEEE Symposium on Information Visualization, 2002. INFOVIS 2002..

[45]  S. Attardo,et al.  Prosodic Markers of Saliency in Humorous Narratives , 2009 .

[46]  Salvatore Attardo,et al.  Prosodic and multimodal markers of humor in conversation , 2011 .

[47]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[48]  Renxian Zhang,et al.  Recognizing Humor on Twitter , 2014, CIKM.

[49]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[50]  Nan Cao,et al.  Introduction to Text Visualization , 2016, Atlantis Briefs in Artificial Intelligence.

[51]  Carlo Strapparava,et al.  Making Computers Laugh: Investigations in Automatic Humor Recognition , 2005, HLT.

[52]  Yong Wang,et al.  VoiceCoach: Interactive Evidence-based Training for Voice Modulation Skills in Public Speaking , 2020, CHI.

[53]  Pascale Fung,et al.  A Long Short-Term Memory Framework for Predicting Humor in Dialogues , 2016, NAACL.

[54]  Mark Liberman,et al.  Speaker identification on the SCOTUS corpus , 2008 .

[55]  Rupal Patel,et al.  ReadN'Karaoke: visualizing prosody in children's books for expressive oral reading , 2011, CHI.

[56]  Md Main Uddin Rony,et al.  ClaimViz: Visual Analytics for Identifying and Verifying Factual Claims , 2020, 2020 IEEE Visualization Conference (VIS).

[57]  Daniel A. Keim,et al.  ConToVi: Multi‐Party Conversation Exploration using Topic‐Space Views , 2016, Comput. Graph. Forum.

[58]  G. Jefferson A technique for inviting laughter and its subsequent acceptance/declination , 1979 .

[59]  Louis-Philippe Morency,et al.  UR-FUNNY: A Multimodal Language Dataset for Understanding Humor , 2019, EMNLP.

[60]  Ben Shneiderman,et al.  Discovering interesting usage patterns in text collections: integrating text mining with visualization , 2007, CIKM '07.

[61]  Alex Endert,et al.  EmoCo: Visual Analysis of Emotion Coherence in Presentation Videos , 2019, IEEE Transactions on Visualization and Computer Graphics.

[62]  Michael Halliday,et al.  An Introduction to Functional Grammar , 1985 .