Trusting Virtual Agents

We present artificial intelligent (AI) agents that act as interviewers to engage with a user in a text-based conversation and automatically infer the user's personality traits. We investigate how the personality of an AI interviewer and the inferred personality of a user influences the user's trust in the AI interviewer from two perspectives: the user's willingness to confide in and listen to an AI interviewer. We have developed two AI interviewers with distinct personalities and deployed them in a series of real-world events. We present findings from four such deployments involving 1,280 users, including 606 actual job applicants. Notably, users are more willing to confide in and listen to an AI interviewer with a serious, assertive personality in a high-stakes job interview. Moreover, users’ personality traits, inferred from their chat text, along with interview context, influence their perception of and their willingness to confide in and listen to an AI interviewer. Finally, we discuss the design implications of our work on building hyper-personalized, intelligent agents.

[1]  Sergio Escalera,et al.  Challenges in multimodal gesture recognition , 2016, J. Mach. Learn. Res..

[2]  Alan R. Wagner,et al.  Overtrust of robots in emergency evacuation scenarios , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Rong Hu,et al.  Enhancing collaborative filtering systems with personality information , 2011, RecSys '11.

[4]  Amy L. Baylor,et al.  The Pedagogical Agent Split-Persona Effect: When Two Agents are Better than One , 2003 .

[5]  P. Costa,et al.  Revised NEO Personality Inventory (NEO-PI-R) and NEO-Five-Factor Inventory (NEO-FFI) , 1992 .

[6]  D. Christopher Dryer,et al.  Getting Personal with Computers: How to Design Personalities for Agents , 1999, Appl. Artif. Intell..

[7]  Jeremy N. Bailenson,et al.  A meta-analysis of the impact of the inclusion and realism of human-like faces on user experiences in interfaces , 2007, CHI.

[8]  Antonio Krüger,et al.  Modelling personality in voices of talking products through prosodic parameters , 2007, IUI '07.

[9]  K. M. Lee,et al.  Can robots manifest personality? : An empirical test of personality recognition, social responses, and social presence in human-robot interaction , 2006 .

[10]  Steven Skiena,et al.  Latent human traits in the language of social media: An open-vocabulary approach , 2017, PloS one.

[11]  Yukiko I. Nakano,et al.  Investigating culture-related aspects of behavior for virtual characters , 2013, Autonomous Agents and Multi-Agent Systems.

[12]  M. Schröder,et al.  Modelling personality features by changing prosody in synthetic speech , 2006 .

[13]  Aaron Powers,et al.  Matching robot appearance and behavior to tasks to improve human-robot cooperation , 2003, The 12th IEEE International Workshop on Robot and Human Interactive Communication, 2003. Proceedings. ROMAN 2003..

[14]  T. Graepel,et al.  Private traits and attributes are predictable from digital records of human behavior , 2013, Proceedings of the National Academy of Sciences.

[15]  J. M. Digman.  PERSONALITY STRUCTURE: EMERGENCE OF THE FIVE-FACTOR MODEL , 1990 .

[16]  Jeffrey D. Ullman,et al.  Introduction to automata theory, languages, and computation, 2nd edition , 2001, SIGA.

[17]  Jeffrey Nichols,et al.  Recommending targeted strangers from whom to solicit information on social media , 2013, IUI '13.

[18]  Etienne de Sevin,et al.  Evaluation of Four Designed Virtual Agent Personalities , 2012, IEEE Transactions on Affective Computing.

[19]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[20]  Jesse Fox,et al.  Inferences About Avatars: Sexism, Appropriateness, Anthropomorphism, and the Objectification of Female Virtual Representations , 2015, J. Comput. Mediat. Commun..

[21]  Barbara F. Okun.  Effective helping: Interviewing and counseling techniques, 5th ed. , 1997 .

[22]  Lyle H. Ungar,et al.  Analyzing Personality through Social Media Profile Picture Choice , 2016, ICWSM.

[23]  Tom Booth,et al.  The Distance Between Mars and Venus: Measuring Global Sex Differences in Personality , 2012, PloS one.

[24]  Alessandro Vinciarelli,et al.  A Survey of Personality Computing , 2014, IEEE Transactions on Affective Computing.

[25]  Corina Yen,et al.  The Man Who Lied to His Laptop: What Machines Teach Us About Human Relationships , 2010 .

[26]  Judith Masthoff,et al.  Adapting Recommendation Diversity to Openness to Experience: A Study of Human Behaviour , 2013, UMAP.

[27]  Sibel Adali,et al.  Predicting Personality with Social Behavior , 2012, 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining.

[28]  Nicu Sebe,et al.  Please, tell me about yourself: automatic personality assessment using short self-presentations , 2011, ICMI '11.

[29]  Jay F. Nunamaker,et al.  Embodied Conversational Agent-Based Kiosk for Automated Interviewing , 2011, J. Manag. Inf. Syst..

[30]  Alan Wexelblat,et al.  Don't Make That Face: a report on anthropomorphizing an interface , 1998 .

[31]  Penelope Brown,et al.  Politeness: Some Universals in Language Usage , 1989 .

[32]  Gerald J. Hahn,et al.  Applied Regression Analysis (2nd Ed.) , 2012 .

[33]  Justine Cassell,et al.  Small Talk and Conversational Storytelling In Embodied Conversational Interface Agents , 1999 .

[34]  Abigail Sellen,et al.  "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents , 2016, CHI.

[35]  John E. Hopcroft,et al.  An n log n algorithm for minimizing states in a finite automaton , 1971 .

[36]  Miroslav Kubat,et al.  Vocabulary Richness Measure in Genres , 2013, J. Quant. Linguistics.

[37]  Denny Britz,et al.  Generating Long and Diverse Responses with Neural Conversation Models , 2017, ArXiv.

[38]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[39]  David L. Hoover,et al.  Another Perspective on Vocabulary Richness , 2003, Comput. Humanit..

[40]  Traum. David,et al.  Computational Approaches to Dialogue , 2017 .

[41]  J. L. Holland,et al.  Making vocational choices : a theory of vocational personalities and work environments , 1984 .

[42]  Robert A. Baron,et al.  Interviewers’ Moods and Evaluations of Job Applicants: The Role of Applicant Qualifications1 , 1993 .

[43]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[44]  Michelle X. Zhou,et al.  KnowMe and ShareMe: understanding automatically discovered personality traits from social media and user sharing preferences , 2014, CHI.

[45]  鈴木 聡.  Media Equation 研究の背景と動向 , 2011 .

[46]  Tobias Baur,et al.  Interpreting social cues to generate credible affective reactions of virtual job interviewers , 2014, ArXiv.

[47]  Adam J. Sporka,et al.  Recognition of Personality Traits from Human Spoken Conversations , 2011, INTERSPEECH.

[48]  Francisco J. Perales López,et al.  Influence of head orientation in perception of personality traits in virtual agents , 2011, AAMAS.

[49]  Joelle Pineau,et al.  Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models , 2015, AAAI.

[50]  De Ayala,et al.  The Theory and Practice of Item Response Theory , 2008 .

[51]  Quoc V. Le,et al.  A Neural Conversational Model , 2015, ArXiv.

[52]  Geoffrey Zweig,et al.  The microsoft 2016 conversational speech recognition system , 2016, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[53]  Dirk Heylen,et al.  First Impressions: Users' Judgments of Virtual Agents' Personality and Interpersonal Attitude in First Encounters , 2012, IVA.

[54]  Tal Yarkoni.  Personality in 100,000 Words: A large-scale analysis of personality and word use among bloggers. , 2010, Journal of research in personality.

[55]  Tom Rath,et al.  Strengths Finder 2.0 , 2007 .

[56]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[57]  M. Ziegler,et al.  New perspectives on faking in personality assessment. , 2011 .

[58]  P. Costa,et al.  NEO inventories for the NEO Personality Inventory-3 (NEO-PI-3), NEO Five-Factor Inventory-3 (NEO-FFI-3), NEO Personality Inventory-Revised (NEO PI-R) : professional manual , 2010 .

[59]  Lee Sproull,et al.  When the Interface Is a Face , 1996, Hum. Comput. Interact..

[60]  Patrick Gebhard,et al.  Exploring interaction strategies for virtual characters to induce stress in simulated job interviews , 2014, AAMAS.

[61]  Jeremy Horpedahl,et al.  Zero to One: Notes on Startups, or How to Build the Future , 2015 .

[62]  Clifford Nass,et al.  Computers are social actors , 1994, CHI '94.

[63]  Martin E. P. Seligman,et al.  The Online Social Self , 2014, Assessment.

[64]  J. Pennebaker,et al.  Linguistic styles: language use as an individual difference. , 1999, Journal of personality and social psychology.

[65]  Peter Lynn,et al.  The effect of interviewer experience, attitudes, personality and skills on respondent co-operation with face-to-face surveys , 2013 .

[66]  Andreas Stolcke,et al.  The Microsoft 2017 Conversational Speech Recognition System , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[67]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[68]  R. Cialdini.  Influence: The Psychology of Persuasion , 1993 .

[69]  Liad Uziel,et al.  Impression management ("lie") scales are associated with interpersonally oriented self-control, not other-deception. , 2014, Journal of personality.

[70]  Adriana Tapus,et al.  User—robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy , 2008, Intell. Serv. Robotics.

[71]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[72]  Lyle H. Ungar,et al.  Studying the Dark Triad of Personality through Twitter Behavior , 2016, CIKM.

[73]  Paul C. Bauer,et al.  Personality traits and the propensity to trust friends and strangers , 2016 .

[74]  Rosalind W. Picard,et al.  Establishing the computer-patient working alliance in automated health behavior change interventions. , 2005, Patient education and counseling.

[75]  B. J. Fogg,et al.  Silicon sycophants: the effects of computers that flatter , 1997, Int. J. Hum. Comput. Stud..

[76]  Tara Matthews,et al.  Can an Algorithm Know the "Real You"?: Understanding People's Reactions to Hyper-personal Analytics Systems , 2015, CHI.

[77]  Jinyan Fan,et al.  Testing the efficacy of a new procedure for reducing faking on personality tests within selection contexts. , 2012, The Journal of applied psychology.

[78]  A. Koller,et al.  Speech Acts: An Essay in the Philosophy of Language , 1969 .

[79]  Alessandro Vinciarelli,et al.  Face-Based Automatic Personality Perception , 2014, ACM Multimedia.

[80]  Jon Oberlander,et al.  What Are They Blogging About? Personality, Topic and Motivation in Blogs , 2009, ICWSM.

[81]  D. Paulhus.  Measurement and control of response bias. , 1991 .

[82]  François Bouchet,et al.  Managing Personality Influences in Dialogical Agents , 2013, ICAART.

[83]  Jeffrey Nichols,et al.  Who will retweet this?: Automatically Identifying and Engaging Strangers on Twitter to Spread Information , 2014, IUI.

[84]  R. Zajonc.  Feeling and thinking : Preferences need no inferences , 1980 .

[85]  Fei Wang,et al.  Who have got answers?: growing the pool of answerers in a smart enterprise social QA system , 2014, IUI.

[86]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[87]  Peter A Thiel,et al.  Zero to One: Notes on Startups, or How to Build the Future , 2016 .

[88]  Pattie Maes,et al.  Agents with Faces: The Effects of Personification of Agents , 1996 .

[89]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[90]  Marilyn A. Walker,et al.  Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text , 2007, J. Artif. Intell. Res..

[91]  L. R. Goldberg,et al.  Integration of the big five and circumplex approaches to trait structure. , 1992, Journal of personality and social psychology.

[92]  Michael E. Roloff,et al.  Strategic emotion in negotiation: Cognition, emotion, and culture , 2012 .

[93]  John E. Laird,et al.  The Soar Cognitive Architecture , 2012 .

[94]  Louis-Philippe Morency,et al.  It's only a computer: Virtual humans increase willingness to disclose , 2014, Comput. Hum. Behav..

[95]  Mehryar Mohri,et al.  Finite-State Transducers in Language and Speech Processing , 1997, CL.

[96]  Anbang Xu,et al.  A New Chatbot for Customer Service on Social Media , 2017, CHI.

[97]  Adriana Tapus,et al.  A model for synthesizing a combined verbal and nonverbal behavior based on personality traits in human-robot interaction , 2013, 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[98]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[99]  Joseph Weizenbaum,et al.  ELIZA—a computer program for the study of natural language communication between man and machine , 1966, CACM.

[100]  D. Ones,et al.  Mechanical versus clinical data combination in selection and admissions decisions: a meta-analysis. , 2013, The Journal of applied psychology.

[101]  Kallirroi Georgila,et al.  SimSensei kiosk: a virtual human interviewer for healthcare decision support , 2014, AAMAS.

[102]  Francesca Odone,et al.  Adaptive Body Gesture Representation for Automatic Emotion Recognition , 2016, ACM Trans. Interact. Intell. Syst..

[103]  Meng-Li Yang,et al.  Does Interviewer Personality Matter for Survey Outcomes?Evidence from a Face-to-face PanelStudy in Taiwan , 2014 .

[104]  Gregor Mehlmann,et al.  Modeling Grounding for Interactive Social Companions , 2015, KI - Künstliche Intelligenz.

[105]  Matthew Turk,et al.  Multimodal interaction: A review , 2014, Pattern Recognit. Lett..

[106]  François Bouchet,et al.  Intelligent Agents with Personality: From Adjectives to Behavioral Schemes , 2012 .

[107]  L. Havens,et al.  Making Contact: Uses of Language in Psychotherapy , 1986 .

[108]  Colin G. DeYoung,et al.  Gender Differences in Personality across the Ten Aspects of the Big Five , 2011, Front. Psychology.

[109]  Subramanian Ramanathan,et al.  Connecting Meeting Behavior with Extraversion—A Systematic Study , 2012, IEEE Transactions on Affective Computing.

[110]  Jordan B Peterson,et al.  Between facets and domains: 10 aspects of the Big Five. , 2007, Journal of personality and social psychology.