How confident are you? Exploring the role of fillers in the automatic prediction of a speaker’s confidence

"Fillers", example "um" in English, have been linked to the "Feeling of Another’s Knowing (FOAK)" or the listener’s perception of a speaker’s expressed confidence. Yet, in Spoken Language Processing (SLP) they remain unexplored, or overlooked as noise. We introduce a new and challenging task, that is the prediction of FOAK, which we think has widespread applicability, given the increasing popularity of automatic processing of educational and job interviews, reviews and speeches. We design a set of filler features based on linguistic literature, and investigate their potential in FOAK prediction. We show that the integration of information related to implicature meanings allows an improvement in the FOAK model and that the different functions of fillers are differently correlated with confidence.

[1]  Anne H. Soukhanov,et al.  The american heritage dictionary of the english language , 1992 .

[2]  Martin Corley,et al.  Hesitation Disfluencies in Spontaneous Speech: The Meaning of um , 2008, Lang. Linguistics Compass.

[3]  Heather Pon-Barry,et al.  Prosodic manifestations of confidence and uncertainty in spoken language , 2008, INTERSPEECH.

[4]  Fabien Ringeval,et al.  Affective and behavioural computing: Lessons learnt from the First Computational Paralinguistics Challenge , 2019, Comput. Speech Lang..

[5]  Dirk Heylen,et al.  Detecting Uncertainty in Spoken Dialogues An explorative research to the automatic detection of a speakers' uncertainty by using prosodic markers , 2011 .

[6]  Divya Saini.  The Effect of Speech Disfluencies on Turn-Taking , 2017 .

[7]  H. H. Clark,et al.  On the Course of Answering Questions , 1993 .

[8]  Siobhan Chapman.  Logic and Conversation , 2005 .

[9]  Dinesh Babu Jayagopi,et al.  Automatic assessment of communication skill in interface-based employment interviews using audio-visual cues , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[10]  Xiaoming Jiang,et al.  The sound of confidence and doubt , 2017, Speech Commun..

[12]  Robin J. Lickley,et al.  Disfluency patterns in dialogue processing , 2010, DiSS-LPSS.

[13]  Alessandro Vinciarelli,et al.  A Survey of Personality Computing , 2014, IEEE Transactions on Affective Computing.

[14]  Chloé Clavel,et al.  A multimodal movie review corpus for fine-grained opinion mining , 2019, ArXiv.

[15]  Elizabeth Shriberg,et al.  Phonetic Consequences of Speech Disfluency , 1999 .

[16]  P. Ekman,et al.  Relative importance of face, body, and speech in judgments of personality and affect. , 1980 .

[17]  Marilyn A. Walker,et al.  Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text , 2007, J. Artif. Intell. Res..

[18]  John J. Godfrey,et al.  SWITCHBOARD: telephone speech corpus for research and development , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  M. Swerts,et al.  Audiovisual prosody and feeling of knowing , 2005 .

[20]  Louis-Philippe Morency,et al.  Computational Analysis of Persuasiveness in Social Multimedia: A Novel Dataset and Multimodal Prediction Approach , 2014, ICMI.

[21]  William Morris.  The American Heritage dictionary of the English language , 1969 .

[22]  Barbara Schuppler,et al.  Automatic detection of uncertainty in spontaneous German dialogue , 2015, INTERSPEECH.

[23]  D. Donaldson,et al.  It’s the way that you, er, say it: Hesitations in speech affect language comprehension , 2007, Cognition.

[24]  S. Brennan,et al.  THE FEELING OF ANOTHER'S KNOWING : PROSODY AND FILLED PAUSES AS CUES TO LISTENERS ABOUT THE METACOGNITIVE STATES OF SPEAKERS , 1995 .

[25]  G. Tottie.  On the use of uh and um in American English , 2014 .

[26]  Mark Liberman,et al.  Speaker identification on the SCOTUS corpus , 2008 .

[27]  H. H. Clark,et al.  Using uh and um in spontaneous speaking , 2002, Cognition.

[28]  Elizabeth Shriberg.  To ‘errrr’ is human: ecology and acoustics of speech disfluencies , 2001, Journal of the International Phonetic Association.

[29]  Esther Le Grézause,et al.  Um and Uh, and the expression of stance in conversational speech , 2017 .

[30]  Pentti Haddington,et al.  Stance Taking in News Interviews , 2004 .