Computational Analysis of Persuasiveness in Social Multimedia: A Novel Dataset and Multimodal Prediction Approach

Our lives are heavily influenced by persuasive communication, and it is essential in almost any types of social interactions from business negotiation to conversation with our friends and family. With the rapid growth of social multimedia websites, it is becoming ever more important and useful to understand persuasiveness in the context of social multimedia content online. In this paper, we introduce our newly created multimedia corpus of 1,000 movie review videos obtained from a social multimedia website called ExpoTV.com, which will be made freely available to the research community. Our research results presented here revolve around the following 3 main research hypotheses. Firstly, we show that computational descriptors derived from verbal and nonverbal behavior can be predictive of persuasiveness. We further show that combining descriptors from multiple communication modalities (audio, text and visual) improve the prediction performance compared to using those from single modality alone. Secondly, we investigate if having prior knowledge of a speaker expressing a positive or negative opinion helps better predict the speaker's persuasiveness. Lastly, we show that it is possible to make comparable prediction of persuasiveness by only looking at thin slices (shorter time windows) of a speaker's behavior.

[1]  Klaus R. Scherer,et al.  The voice of confidence: Paralinguistic cues and audience evaluation. , 1973 .

[2]  Stephen Worchel,et al.  Is the Medium the Message? A Study of the Effects of Media, Communicator, and Message Characteristics on Attitude Change1 , 1975 .

[3]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[4]  Franziska Marquart,et al.  Communication and persuasion : central and peripheral routes to attitude change , 1988 .

[5]  Katrin Baumgartner,et al.  Persuasion Theory And Research , 2016 .

[6]  A. Pentland,et al.  Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. , 2007, The Journal of applied psychology.

[7]  D. O’Keefe.  Persuasion , 1990, The Handbook of Communication Skills.

[8]  Daniel Gatica-Perez,et al.  FaceTube: predicting personality from facial expressions of emotion in online conversational video , 2012, ICMI '12.

[9]  Samuel E. Casselberry,et al.  Linguistics: Silent Messages. ALBERT MEHRABIAN. Belmont, CA: Wadsworth , 1973 .

[10]  Jeffery Pittam,et al.  The Relationship between Perceived Persuasiveness of Nasality and Source Characteristics for Australian and American Listeners , 1990 .

[11]  W. Crano,et al.  Attitudes and persuasion. , 2006, Annual review of psychology.

[12]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[13]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[14]  Javier R. Movellan,et al.  Generalized adaptive view-based appearance model: Integrated framework for monocular head pose estimation , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[15]  Jakob D. Jensen,et al.  The Relative Persuasiveness of Gain-Framed Loss-Framed Messages for Encouraging Disease Prevention Behaviors: A Meta-Analytic Review , 2007, Journal of health communication.

[16]  Ailbhe Ní Chasaide,et al.  The role of voice quality in communicating emotion, mood and attitude , 2003, Speech Commun..

[17]  W. Barnett Pearce,et al.  Vocalic communication in persuasion , 1972 .

[18]  S. Chaiken,et al.  Communication modality as a determinant of message persuasiveness and message comprehensibility. , 1976 .

[19]  John Kane,et al.  COVAREP — A collaborative voice analysis repository for speech technologies , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  John Kane,et al.  Speaker and language independent voice quality classification applied to unlabelled corpora of expressive speech , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  M. B. LaCrosse.  Nonverbal behavior and perceived counselor attractiveness and persuasiveness. , 1975 .

[22]  James Price Dillard,et al.  The Persuasion Handbook: Developments in Theory and Practice , 2002 .

[23]  Chris Arnold,et al.  Language and Persuasion , 2012 .

[24]  Volume Assp,et al.  ACOUSTICS. SPEECH. AND SIGNAL PROCESSING , 1983 .

[25]  Joel Young,et al.  A Microtext Corpus for Persuasion Detection in Dialog , 2011, Analyzing Microtext.

[26]  J. Burgoon,et al.  Nonverbal Behaviors, Persuasion, and Credibility , 1990 .

[27]  Björn W. Schuller,et al.  The INTERSPEECH 2011 Speaker State Challenge , 2011, INTERSPEECH.

[28]  N. Miller,et al.  Speed of speech and persuasion. , 1976 .

[29]  Chanthika Pornpitakpan.  The Persuasiveness of Source Credibility: A Critical Review of Five Decades' Evidence , 2004 .

[30]  Carlos Busso,et al.  Analysis of Emotionally Salient Aspects of Fundamental Frequency for Emotion Detection , 2009, IEEE Transactions on Audio, Speech, and Language Processing.

[31]  H. Rosenfeld,et al.  Approval-seeking and approval-inducing functions of verbal and nonverbal responses in the dyad. , 1966 .

[32]  K. Krippendorff.  Krippendorff, Klaus, Content Analysis: An Introduction to its Methodology . Beverly Hills, CA: Sage, 1980. , 1980 .

[33]  O. John,et al.  Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German , 2007 .

[34]  Siddharth Suri,et al.  Conducting behavioral research on Amazon’s Mechanical Turk , 2010, Behavior research methods.

[35]  John Kane,et al.  A comparative study of glottal open quotient estimation techniques , 2013, INTERSPEECH.

[36]  Alice H. Eagly,et al.  Heuristic and systematic information processing within and beyond the persuasion context. , 1989 .

[37]  Yiming Yang,et al.  A Comparative Study on Feature Selection in Text Categorization , 1997, ICML.

[38]  Steven E Stern,et al.  Effects of perceived disability on persuasiveness of computer-synthesized speech. , 2002, The Journal of applied psychology.

[39]  N. Ambady,et al.  Thin slices of expressive behavior as predictors of interpersonal consequences: A meta-analysis. , 1992 .

[40]  Joan Meyers-Levy,et al.  Consumers' processing of persuasive advertisements: An integrative framework of persuasion theories , 1999 .

[41]  A. Mehrabian,et al.  Nonverval concomitants of perceived and intended persuasiveness. , 1969, Journal of personality and social psychology.

[42]  Friedhelm Schwenker,et al.  Investigating fuzzy-input fuzzy-output support vector machines for robust voice quality classification , 2013, Comput. Speech Lang..

[43]  G. Mahl.  Disturbances and silences in the patient's speech in psychotherapy. , 1956, Journal of abnormal psychology.