Design Principles for Special Purpose, Embodied, Conversational Intelligence with Environmental Sensors (SPECIES) Agents

As information systems increase their ability to gather and analyze data from the natural environment and as computational power increases, the next generation of human-computer interfaces will be able to facilitate more lifelike and natural interactions with humans. This can be accomplished by using sensors to non-invasively gather information from the user, using artificial intelligence to interpret this information to perceive users’ emotional and cognitive states, and using customized interfaces and responses based on embodied-conversational-agent (avatar) technology to respond to the user. We refer to this novel and unique class of intelligent agents as Special Purpose Embodied Conversational Intelligence with Environmental Sensors (SPECIES) agents. In this paper, we build on interpersonal communication theory to specify four essential design principles of all SPECIES agents. We also share findings of initial research that demonstrates how SPECIES agents can be deployed to augment human tasks. Results of this paper organize future research efforts in collectively studying and creating more robust, influential, and intelligent SPECIES agents.

[1]  Harri Oinas-Kukkonen,et al.  Persuasive Systems Design: Key Issues, Process Model, and System Features , 2009, Commun. Assoc. Inf. Syst..

[2]  A. Vrij.  Detecting Lies and Deceit: Pitfalls and Opportunities , 2008 .

[3]  Jennifer Edson Escalas.  Self-Referencing and Persuasion: Narrative Transportation versus Analytical Elaboration , 2007 .

[4]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[5]  Kathy Kellermann.  Communication: Inherently strategic and primarily automatic , 1992 .

[6]  Richard E Petty,et al.  Thought confidence as a determinant of persuasion: the self-validation hypothesis. , 2002, Journal of personality and social psychology.

[7]  Eric T. Bradlow,et al.  A Feature-Based Approach to Assessing Advertisement Similarity , 2006 .

[8]  David R. Firth,et al.  Communications of the Association for Information Systems , 2011 .

[9]  Ritu Agarwal,et al.  Research Note - Social Interactions and the "Digital Divide": Explaining Variations in Internet Use , 2009, Inf. Syst. Res..

[10]  S. Asch.  Effects of Group Pressure Upon the Modification and Distortion of Judgments , 1951 .

[11]  H. Kelley,et al.  Communication and Persuasion: Psychological Studies of Opinion Change , 1982 .

[12]  Shelly Chaiken,et al.  The pursuit of self-interest: self-interest bias in attitude judgment and persuasion. , 2005, Journal of personality and social psychology.

[13]  Joseph P. Simmons,et al.  Branding Alters Attitude Functions and Reduces the Advantage of Function-Matching Persuasive Appeals , 2007 .

[14]  H. Raghav Rao,et al.  Protection motivation and deterrence: a framework for security policy compliance in organisations , 2009, Eur. J. Inf. Syst..

[15]  Kenneth G. DeBono,et al.  Source expertise, source attractiveness, and the processing of persuasive information: A functional approach. , 1988 .

[16]  James J. Lindsay,et al.  Cues to deception. , 2003, Psychological bulletin.

[17]  B. J. Fogg,et al.  The elements of computer credibility , 1999, CHI '99.

[18]  Judee K. Burgoon,et al.  The ideal source: A reexamination of source credibility measurement , 1976 .

[19]  Mikko Sams,et al.  The effect of dynamics on identifying basic emotions from synthetic and natural faces , 2008, Int. J. Hum. Comput. Stud..

[20]  Donald F. Roberts,et al.  The Process and Effects of Mass Communication Revised Edition , 1971 .

[21]  Matthew L. Jensen,et al.  Effects of Automated and Participative Decision Support in Computer-Aided Credibility Assessment , 2009, J. Manag. Inf. Syst..

[22]  Mike Thelwall,et al.  The Dimensions of Web Site Credibility and Their Relation to Active Trust and Behavioural Impact , 2009, Commun. Assoc. Inf. Syst..

[23]  Dvijesh Shastri,et al.  Imaging Facial Physiology for the Detection of Deceit , 2006, International Journal of Computer Vision.

[24]  Ioannis Pavlidis,et al.  Human behaviour: Seeing through the face of deception , 2002, Nature.

[25]  Uma R. Karmarkar,et al.  Believe Me, I Have No Idea What I’m Talking About: The Effects of Source Certainty on Consumer Involvement and Persuasion , 2010 .

[26]  Ning Wang,et al.  The politeness effect: Pedagogical agents and learning outcomes , 2008, Int. J. Hum. Comput. Stud..

[27]  G. Bente,et al.  Avatar-Mediated Networking: Increasing Social Presence and Interpersonal Trust in Net-Based Collaborations , 2008 .

[28]  W. Feek.  Communication works. , 1996, AIDS/STD health promotion exchange.

[29]  S. Asch.  Studies of independence and conformity: I. A minority of one against a unanimous majority. , 1956 .

[30]  Kensuke Terai,et al.  Respiratory changes during detection of deception , 2001 .

[31]  Judee K. Burgoon,et al.  The voice of deceit: Refining and expanding vocal cues to deception , 1997 .

[32]  Mark W. Patton.  DECISION SUPPORT FOR RAPID ASSESSMENT OF TRUTH AND DECEPTION USING AUTOMATED ASSESSMENT TECHNOLOGIES AND KIOSK-BASED EMBODIED CONVERSATIONAL AGENTS , 2009 .

[33]  S. Asch.  Opinions and Social Pressure , 1955, Nature.

[34]  Paul Benjamin Lowry,et al.  Explaining and Predicting the Impact of Branding Alliances and Web Site Quality on Initial Consumer Trust of E-Commerce Web Sites , 2007, J. Manag. Inf. Syst..

[35]  Mitsuru Ishizuka,et al.  Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game , 2005, Int. J. Hum. Comput. Stud..

[36]  T. Gutheil.  Detecting Lies and Deceit: Pitfalls and Opportunities , 2010 .

[37]  C. F. Kao,et al.  Central and peripheral routes to persuasion: An individual difference perspective. , 1986 .

[38]  W. Wood.  Attitude change: persuasion and social influence. , 2000, Annual review of psychology.

[39]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[40]  Derek D. Rucker,et al.  Unpacking attitude certainty: attitude clarity and attitude correctness. , 2007, Journal of personality and social psychology.

[41]  K. Stewart.  Trust Transfer on the World Wide Web , 2002 .

[42]  I. Pavlov,et al.  [Conditioned reflex]. , 1951, Fel'dsher i akusherka.

[43]  Jeffrey M. Bradshaw,et al.  Ten Challenges for Making Automation a "Team Player" in Joint Human-Agent Activity , 2004, IEEE Intell. Syst..

[44]  Richard E. Mayer,et al.  Constructing computer-based tutors that are socially sensitive: Politeness in educational software , 2006, Int. J. Hum. Comput. Stud..

[45]  Ritu Agarwal,et al.  Adoption of Electronic Health Records in the Presence of Privacy Concerns: The Elaboration Likelihood Model and Individual Persuasion , 2009, MIS Q..

[46]  Jörn Ostermann,et al.  User evaluation: Synthetic talking faces for interactive services , 1999, The Visual Computer.

[47]  Sid L. Huff,et al.  CIO lateral influence behaviors: gaining peers' commitment to strategic information systems , 2000, ICIS.

[48]  R. E. Lubow,et al.  Pupillary size in response to a visual guilty knowledge test: New technique for the detection of deception , 1996 .

[49]  Robert O. Briggs,et al.  The Yield Shift Theory of Satisfaction and Its Application to the IS/IT Domain , 2008, J. Assoc. Inf. Syst..

[50]  Ying Liu,et al.  An evaluation of the chat and knowledge delivery components of a low-level dialog system: The AZ-ALICE experiment , 2007, Decis. Support Syst..

[51]  R. Petty,et al.  Positive Mood and Persuasion: Different Roles for Affect Under High- and Low-Elaboration Conditions , 1993 .

[52]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[53]  Edward J. Garrity,et al.  An experimental investigation of web-based information systems success in the context of electronic commerce , 2005, Decis. Support Syst..

[54]  E. N. Solokov.  Perception and the conditioned reflex , 1963 .

[55]  Frank Marchak,et al.  Credibility assessments: operational issues and technology impact for law enforcement applications , 2003, SPIE Defense + Commercial Sensing.

[56]  Norman I. Badler,et al.  Creating Interactive Virtual Humans: Some Assembly Required , 2002, IEEE Intell. Syst..

[57]  B. J. Fogg,et al.  Persuasive technology: using computers to change what we think and do , 2002, UBIQ.

[58]  Stacy Marsella,et al.  Virtual Rapport , 2006, IVA.

[59]  Jay F. Nunamaker,et al.  Border Security Credibility Assessments via Heterogeneous Sensor Fusion , 2010, IEEE Intelligent Systems.

[60]  Ioannis Pavlidis,et al.  The face of fear , 2001 .

[61]  Dale Goodhue,et al.  Task-Technology Fit and Individual Performance , 1995, MIS Q..

[62]  J. Cacioppo,et al.  Attitudes and Persuasion: Classic and Contemporary Approaches , 1981 .

[63]  John P. Lewis,et al.  Perceiving Visual Emotions with Speech , 2006, IVA.

[64]  Jay F. Nunamaker,et al.  Automated Kinesic Analysis for Deception Detection , 2010 .

[65]  James D. Westphal,et al.  FLATTERY WILL GET YOU EVERYWHERE (ESPECIALLY IF YOU ARE A MALE CAUCASIAN): HOW INGRATIATION, BOARDROOM BEHAVIOR, AND DEMOGRAPHIC MINORITY STATUS AFFECT ADDITIONAL BOARD APPOINTMENTS AT U.S. COMPANIES , 2007 .

[66]  Matthew L. Jensen,et al.  Automatic Extraction of Deceptive Behavioral Cues from Video , 2005, ISI.

[67]  Aaron C. Elkins,et al.  Evaluating the Credibility Assessment Capability of Vocal Analysis Software , 2012 .

[68]  G. Bower.  Mood and memory. , 1981, The American psychologist.

[69]  H Stanislaw,et al.  Calculation of signal detection theory measures , 1999, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[70]  Ira Rudowsky,et al.  Intelligent Agents , 2004, Commun. Assoc. Inf. Syst..

[71]  J. Forgas.  Mood and judgment: the affect infusion model (AIM). , 1995, Psychological bulletin.

[72]  G. Clore,et al.  Mood, misattribution, and judgments of well-being: Informative and directive functions of affective states. , 1983 .

[73]  A. Meystel,et al.  Intelligent Systems , 2001 .

[74]  J. Cacioppo,et al.  Effects of message repetition and position on cognitive response, recall, and persuasion. , 1979 .

[75]  Mervyn A. Jack,et al.  Evaluating humanoid synthetic agents in e-retail applications , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[76]  Kenton O'Hara,et al.  Social Impact , 2019, Encyclopedia of Food and Agricultural Ethics.

[77]  Daniel M. Cable,et al.  Managers' upward influence tactic strategies: the role of manager personality and supervisor leadership style , 2003 .

[78]  Chin-Lung Hsu,et al.  Acceptance of blog usage: The roles of technology acceptance, social influence and knowledge sharing motivation , 2008, Inf. Manag..

[79]  David Woods,et al.  1. How to make automated systems team players , 2002 .

[80]  Richard E Petty,et al.  The effects of message recipients' power before and after persuasion: a self-validation analysis. , 2007, Journal of personality and social psychology.