SemEval-2020 Task 7: Assessing Humor in Edited News Headlines

This paper describes the SemEval-2020 shared task "Assessing Humor in Edited News Headlines." The task's dataset contains news headlines in which short edits were applied to make them funny, and the funniness of these edited headlines was rated using crowdsourcing. This task includes two subtasks, the first of which is to estimate the funniness of headlines on a humor scale in the interval 0-3. The second subtask is to predict, for a pair of edited versions of the same original headline, which is the funnier version. To date, this task is the most popular shared computational humor task, attracting 48 teams for the first subtask and 31 teams for the second.

[1]  Mahmoud Hammad,et al.  MLEngineer at SemEval-2020 Task 7: BERT-Flair Based Humor Detection Model (BFHumor) , 2020, SemEval@COLING.

[2]  Xuejie Zhang,et al.  YNU-HPCC at SemEval-2020 Task 7: Using an Ensemble BiGRU Model to Evaluate the Humor of Edited News Titles , 2020, SemEval@COLING.

[3]  Alon Rozental,et al.  Amobee at SemEval-2020 Task 7: Regularization of Language Model Based Classifiers , 2020, SemEval@COLING.

[4]  Yishay Raz,et al.  Automatic Humor Classification on Twitter , 2012, NAACL.

[5]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[6]  Rajaswa Patil,et al.  LRG at SemEval-2020 Task 7: Assessing the Ability of BERT and Derivative Models to Perform Short-Edits Based Humor Grading , 2020, SEMEVAL.

[7]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[8]  Enas Khwaileh,et al.  ELMo-NB at SemEval-2020 Task 7: Assessing Sense of Humor in EditedNews Headlines Using ELMo and NB , 2020, SemEval@COLING.

[9]  Jian Ma,et al.  XSYSIGMA at SemEval-2020 Task 7: Method for Predicting Headlines’ Humor Based on Auxiliary Sentences with EI-BERT , 2020, SEMEVAL.

[10]  Masaki Aono,et al.  KDEhumor at SemEval-2020 Task 7: A Neural Network Model for Detecting Funniness in Dataset Humicroedit , 2020, SemEval@COLING.

[11]  Cheng Zhang,et al.  WUY at SemEval-2020 Task 7: Combining BERT and Naive Bayes-SVM for Humor Assessment in Edited News Headlines , 2020, SEMEVAL.

[12]  Yue Yin,et al.  Duluth at SemEval-2020 Task 7: Using Surprise as a Key to Unlock Humorous Headlines , 2020, SEMEVAL.

[13]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[14]  Anna Rumshisky,et al.  SemEval-2017 Task 6: #HashtagWars: Learning a Sense of Humor , 2017, *SEMEVAL.

[15]  Carlo Strapparava,et al.  Making Computers Laugh: Investigations in Automatic Humor Recognition , 2005, HLT.

[16]  Man Lan,et al.  ECNU at SemEval-2020 Task 7: Assessing Humor in Edited News Headlines Using BiLSTM with Attention , 2020, SemEval@COLING.

[17]  Gustavo Henrique Paetzold.  UTFPR at SemEval-2020 Task 7: Using Co-occurrence Frequencies to Capture Unexpectedness , 2020, SemEval@COLING.

[18]  Anita Soloveva.  SO at SemEval-2020 Task 7: DeepPavlov Logistic Regression with BERT Embeddings vs SVR at Funniness Evaluation , 2020, SemEval@COLING.

[19]  Walid Magdy,et al.  Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating , 2020, SemEval@COLING.

[20]  Pavel Smrz,et al.  JokeMeter at SemEval-2020 Task 7: Convolutional humor , 2020, SemEval@COLING.

[21]  Eric Horvitz,et al.  Filling the Blanks (hint: plural noun) for Mad Libs Humor , 2017, EMNLP.

[22]  Michael Gamon,et al.  “President Vows to Cut Hair”: Dataset and Analysis of Creative Text Editing for Humorous Headlines , 2019, NAACL.

[23]  Rada Mihalcea,et al.  “Judge me by my size (noun), do you?” YodaLib: A Demographic-Aware Humor Generation Framework , 2020, COLING.

[24]  Iryna Gurevych,et al.  SemEval-2017 Task 7: Detection and Interpretation of English Puns , 2017, *SEMEVAL.

[25]  J. Shaw,et al.  Philosophy of Humor , 2010 .

[26]  Véronique Hoste,et al.  SemEval-2018 Task 3: Irony Detection in English Tweets , 2018, *SEMEVAL.

[27]  Smita Ghaisas,et al.  Hasyarasa at SemEval-2020 Task 7: Quantifying Humor as Departure from Expectedness , 2020, SemEval@COLING.

[28]  Yanru Zhang,et al.  Ferryman at SemEval-2020 Task 7: Ensemble Model for Assessing Humor in Edited News Headlines , 2020, SemEval@COLING.

[29]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[30]  Pascale Fung,et al.  A Long Short-Term Memory Framework for Predicting Humor in Dialogues , 2016, NAACL.

[31]  Horacio Saggion,et al.  Automatic Detection of Irony and Humour in Twitter , 2014, ICCC.

[32]  Charlotte Ammer,et al.  UniTuebingenCL at SemEval-2020 Task 7: Humor Detection in News Headlines , 2020, SemEval@COLING.

[33]  Henry A. Kautz,et al.  Stimulating Creativity with FunLines: A Case Study of Humor Generation in Headlines , 2020, ACL.

[34]  Elena Filatova,et al.  Irony and Sarcasm: Corpus Generation and Analysis Using Crowdsourcing , 2012, LREC.

[35]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[36]  Kayalvizhi S,et al.  SSN_NLP at SemEval-2020 Task 7: Detecting Funniness Level Using Traditional Learning with Sentence Embeddings , 2020, SEMEVAL.

[37]  Barbara Plank,et al.  Buhscitu at SemEval-2020 Task 7: Assessing Humour in Edited News Headlines Using Hand-Crafted Features and Online Knowledge Bases , 2020, SemEval@COLING.

[38]  Els Lefever,et al.  LT3 at SemEval-2020 Task 7: Comparing Feature-Based and Transformer-Based Approaches to Detect Funny Headlines , 2020, SemEval@COLING.

[39]  Yuriy Brun,et al.  That's What She Said: Double Entendre Identification , 2011, ACL.

[40]  Hiroaki Ozaki,et al.  Hitachi at SemEval-2020 Task 7: Stacking at Scale with Heterogeneous Language Models for Humor Recognition , 2020, SEMEVAL.

[41]  Anna-Katharina Dick,et al.  HumorAAC at SemEval-2020 Task 7: Assessing the Funniness of Edited News Headlines through Regression and Trump Mentions , 2020, SemEval@COLING.

[42]  Kuan Tang,et al.  Funny3 at SemEval-2020 Task 7: Humor Detection of Edited Headlines with LSTM and TFIDF Neural Network System , 2020, SemEval@COLING.

[43]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[44]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[45]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[46]  Pramodith Ballapuram.  LMML at SemEval-2020 Task 7: Siamese Transformers for Rating Humor in Edited News Headlines , 2020, SemEval@COLING.

[47]  Luis Chiruzzo,et al.  Overview of HAHA at IberLEF 2019: Humor Analysis based on Human Annotation , 2019, IberLEF@SEPLN.

[48]  Davide Buscaldi,et al.  From humor recognition to irony detection: The figurative language of social media , 2012, Data Knowl. Eng..

[49]  Renxian Zhang,et al.  Recognizing Humor on Twitter , 2014, CIKM.