Predicting TED Talk Ratings from Language and Prosody

We use the largest open repository of public speaking---TED Talks---to predict the ratings of the online viewers. Our dataset contains over 2200 TED Talk transcripts (includes over 200 thousand sentences), audio features and the associated meta information including about 5.5 Million ratings from spontaneous visitors of the website. We propose three neural network architectures and compare with statistical machine learning. Our experiments reveal that it is possible to predict all the 14 different ratings with an average AUC of 0.83 using the transcripts and prosody features only. The dataset and the complete source code is available for further analysis.

[1]  Lei Chen,et al.  Convolutional Neural Network for Humor Recognition , 2017, ArXiv.

[2]  Andrei Popescu-Belis,et al.  Sentiment analysis of user comments for one-class collaborative filtering over ted talks , 2013, SIGIR.

[3]  Hao Wang,et al.  Using Argument-based Features to Predict and Analyse Review Helpfulness , 2017, EMNLP.

[4]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[5]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[6]  Helen Yannakoudakis,et al.  Automatic Text Scoring Using Neural Networks , 2016, ACL.

[7]  Jalal Mahmud,et al.  Fostering User Engagement: Rhetorical Devices for Applause Generation Learnt from TED Talks , 2017, ICWSM.

[8]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[9]  Anmol Madan,et al.  SOCIAL SIGNALING: PREDICTING THE OUTCOME OF JOB INTERVIEWS FROM VOCAL TONE AND PROSODY , 2009 .

[10]  Mari Ostendorf,et al.  Phonological Pun-derstanding , 2016, NAACL.

[11]  Hwee Tou Ng,et al.  A Neural Approach to Automated Essay Scoring , 2016, EMNLP.

[12]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[13]  Carl Vogel,et al.  Visual, Laughter, Applause and Spoken Expression Features for Predicting Engagement Within TED Talks , 2017, INTERSPEECH.

[14]  Pascale Fung,et al.  A Long Short-Term Memory Framework for Predicting Humor in Dialogues , 2016, NAACL.

[15]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Jun Zhou,et al.  Cross-Domain Review Helpfulness Prediction Based on Convolutional Neural Networks with Auxiliary Domain Discriminators , 2018, NAACL.

[18]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[19]  Ben He,et al.  TDNN: A Two-stage Deep Neural Network for Prompt-independent Automated Essay Scoring , 2018, ACL.

[20]  Forrest Sheng Bao,et al.  Semantic Analysis and Helpfulness Prediction of Text for Online Product Reviews , 2015, ACL.

[21]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Ted Briscoe,et al.  Neural Automated Essay Scoring and Coherence Modeling for Adversarially Crafted Input , 2018, NAACL.

[24]  Ji Liu,et al.  Unsupervised Extraction of Human-Interpretable Nonverbal Behavioral Cues in a Public Speaking Scenario , 2015, ACM Multimedia.

[25]  Daniel Gildea,et al.  Automated Analysis and Prediction of Job Interview Performance , 2015, IEEE Transactions on Affective Computing.

[26]  Pearl Pu,et al.  Prediction of Helpful Reviews Using Emotions Extraction , 2014, AAAI.

[27]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[28]  Björn W. Schuller,et al.  Words that Fascinate the Listener: Predicting Affective Ratings of On-Line Lectures , 2013, Int. J. Distance Educ. Technol..

[29]  Slav Petrov,et al.  Globally Normalized Transition-Based Neural Networks , 2016, ACL.

[30]  Daniel Gatica-Perez,et al.  Hirability in the Wild: Analysis of Online Conversational Video Resumes , 2016, IEEE Transactions on Multimedia.

[31]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[32]  Xiaoming Xi,et al.  Automatic scoring of non-native spontaneous speech in tests of spoken English , 2009, Speech Commun..

[33]  Raiyan Abdul Baten,et al.  Awe the Audience: How the Narrative Trajectories Affect Audience Perception in Public Speaking , 2018, CHI.

[34]  Salvatore Valenti,et al.  An Overview of Current Research on Automated Essay Grading , 2003, J. Inf. Technol. Educ..

[35]  Daniel Jurafsky,et al.  It’s Not You, it’s Me: Detecting Flirting and its Misperception in Speed-Dates , 2009, EMNLP.

[36]  Richard Socher,et al.  Regularizing and Optimizing LSTM Language Models , 2017, ICLR.

[37]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[38]  V. Vapnik,et al.  A note one class of perceptrons , 1964 .