Cisco at AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides using Contextualised Embeddings

This paper describes our proposed system for the AAAICAD21 shared task: Predicting Emphasis in Presentation Slides. In this specific task, given the contents of a slide we are asked to predict the degree of emphasis to be laid on each word in the slide. We propose 2 approaches to this problem including a BiLSTM-ELMo approach and a transformers based approach based on RoBERTa and XLNet architectures. We achieve a score of 0.518 on the evaluation leaderboard which ranks us 3 and 0.543 on the post-evaluation leaderboard which ranks us 1 at the time of writing the paper.

[1]  Xuanjing Huang,et al.  Keyphrase Extraction Using Deep Recurrent Neural Networks on Twitter , 2016, EMNLP.

[2]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[3]  Rajiv Ratn Shah,et al.  MIDAS at SemEval-2020 Task 10: Emphasis Selection using Label Distribution Learning and Contextual Embeddings , 2020, SemEval@COLING.

[4]  Rong Pan,et al.  Automatic Emphatic Information Extraction from Aligned Acoustic Data and Its Application on Sentence Compression , 2017, AAAI.

[5]  Francisco C. Pereira,et al.  Deep learning from crowds , 2017, AAAI.

[6]  Yu Sun,et al.  ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model , 2020, SEMEVAL.

[7]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[8]  Hinrich Schütze,et al.  Active Learning with Amazon Mechanical Turk , 2011, EMNLP.

[9]  Jie Yang,et al.  Leveraging Crowdsourcing Data for Deep Active Learning An Application: Learning Intents in Alexa , 2018, WWW.

[10]  Taniya Mishra,et al.  Word Prominence Detection using Robust yet Simple Prosodic Features , 2012, INTERSPEECH.

[11]  Franck Dernoncourt,et al.  Learning Emphasis Selection for Written Text in Visual Media from Crowd-Sourced Label Distributions , 2019, ACL.

[12]  Xin Geng,et al.  Label Distribution Learning , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[13]  Isabelle Augenstein,et al.  SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications , 2017, *SEMEVAL.

[14]  Rishabh Agarwal,et al.  IITK at SemEval-2020 Task 10: Transformers for Emphasis Selection , 2020, SemEval@COLING.

[15]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[16]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[17]  Franck Dernoncourt,et al.  Learning to Emphasize: Dataset and Shared Task Models for Selecting Emphasis in Presentation Slides , 2021, ArXiv.

[18]  Franck Dernoncourt,et al.  Let Me Choose: From Verbal Context to Font Selection , 2020, ACL.

[19]  Roger Zimmermann,et al.  Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings , 2019, ArXiv.

[20]  Franck Dernoncourt,et al.  SemEval-2020 Task 10: Emphasis Selection for Written Text in Visual Media , 2020, SEMEVAL.

[21]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[22]  Bernardete Ribeiro,et al.  Sequence labeling with multiple annotators , 2013, Machine Learning.

[23]  Hai Zhao,et al.  Hierarchical Contextualized Representation for Named Entity Recognition , 2019, AAAI.