Understanding algorithm aversion: When is advice from automation discounted?

Forecasting advice from human advisors is often utilized more than advice from automation. There is little understanding of why “algorithm aversion” occurs, or specific conditions that may exaggerate it. This paper first reviews literature from two fields—interpersonal advice and human–automation trust—that can inform our understanding of the underlying causes of the phenomenon. Then, an experiment is conducted to search for these underlying causes. We do not replicate the finding that human advice is generally utilized more than automated advice. However, after receiving bad advice, utilization of automated advice decreased significantly more than advice from humans. We also find that decision makers describe themselves as having much more in common with human than automated advisors despite there being no interpersonal relationship in our study. Results are discussed in relation to other findings from the forecasting and human–automation trust fields and provide a new perspective on what causes and exaggerates algorithm aversion.

[1]  Joseph P. Simmons,et al.  Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them , 2016, Manag. Sci..

[2]  Erina L. MacGeorge,et al.  Advise with Permission?: The Effects of Advice Solicitation on Advice Outcomes , 2017 .

[3]  Bo Feng,et al.  Testing Advice Response Theory in Interactions With Friends , 2016, Commun. Res..

[4]  Erina L. MacGeorge,et al.  Advice: Expanding the Communication Paradigm , 2016 .

[5]  Pouyan Esmaeilzadeh,et al.  Adoption of clinical decision support systems in a developing country: Antecedents and outcomes of physician's threat to perceived professional autonomy , 2015, Int. J. Medical Informatics.

[6]  Masooda Bashir,et al.  Trust in Automation , 2015, Hum. Factors.

[7]  Berkeley J. Dietvorst,et al.  Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err , 2014, Journal of experimental psychology. General.

[8]  David Gefen,et al.  Understanding Trust in IT Artifacts: An Evaluation of the impact of Trustworthiness and Trust on Satisfaction with Antiviral Software , 2014, DATB.

[9]  Ewart J. de Visser,et al.  Human Trust in Other Humans, Automation, Robots, and Cognitive Agents , 2014 .

[10]  Jorge A. Alvarado-Valencia,et al.  Reliance, trust and heuristics in judgmental forecasting , 2014, Comput. Hum. Behav..

[11]  Franklin Dexter,et al.  Review of Experimental Studies in Social Psychology of Small Groups When an Optimal Choice Exists and Application to Operating Room Management Decision-Making , 2013, Anesthesia and analgesia.

[12]  Bo Feng,et al.  Examining Cultural Similarities and Differences in Responses to Advice , 2013, Commun. Res..

[13]  Stavros Asimakopoulos,et al.  Forecasting support systems technologies-in-practice: A model of adoption and use for product forecasting , 2013 .

[14]  Robert Fildes,et al.  Forecasting support systems: What we know, what we need to know , 2013 .

[15]  Paul Goodwin,et al.  Antecedents and effects of trust in forecasting advice , 2013 .

[16]  Takayuki Kanda,et al.  Do people hold a humanoid robot morally accountable for the harm it causes? , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[17]  Van Swol,et al.  Forecasting another’s enjoyment versus giving the right answer: Trust, shared values, task effects, and confidence in improving the acceptance of advice , 2011 .

[18]  E. Jonas,et al.  Forecasting clients' reactions: How does the perception of strategic behavior influence the acceptance of advice? , 2011 .

[19]  Bo Feng,et al.  The Influences of Message and Source Factors on Advice Outcomes , 2010, Commun. Res..

[20]  Lyn M. Van Swol,et al.  The Effects of Confidence and Advisor Motives on Advice Utilization , 2009, Commun. Res..

[21]  Mary E. Thomson,et al.  The relative influence of advice from human experts and statistical methods on forecast adjustments , 2009 .

[22]  R. Fildes,et al.  Effective forecasting and judgmental adjustments: an empirical evaluation and strategies for improvement in supply-chain planning , 2009 .

[23]  Daantje Derks,et al.  The role of emotion in computer-mediated communication: A review , 2008, Comput. Hum. Behav..

[24]  Douglas A. Wiegmann,et al.  Effects of Information Source, Pedigree, and Reliability on Operator Interaction With Decision Support Systems , 2007, Hum. Factors.

[25]  Konstantinos Nikolopoulos,et al.  The process of using a forecasting support system , 2007 .

[26]  D. Wiegmann,et al.  Similarities and differences between human–human and human–automation trust: an integrative review , 2007 .

[27]  J. H. Davis,et al.  An Integrative Model of Organizational Trust: Past, Present, and Future , 2007 .

[28]  S. Bonaccio,et al.  Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences , 2006 .

[29]  Takayuki Kanda,et al.  What is a Human? - Toward Psychological Benchmarks in the Field of Human-Robot Interaction , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[30]  Gunnar E. Schrah,et al.  No decision‐maker is an Island: integrating expert advice with information acquisition , 2006 .

[31]  P. Goodwin,et al.  Judgmental forecasting: A review of progress over the last 25 years , 2006 .

[32]  Lyn M Van Swol,et al.  Factors affecting the acceptance of expert advice. , 2005, The British journal of social psychology.

[33]  Gunnar E. Schrah,et al.  Improving judgement with prepaid expert advice , 2004 .

[34]  Detmar W. Straub,et al.  Trust and TAM in Online Shopping: An Integrated Model , 2003, MIS Q..

[35]  S. Hoffman.  Unmanaged care: towards moral fairness in health care coverage. , 2002, Indiana law journal.

[36]  Nanda Izak Kumar,et al.  Para-Social Presence and Communication Capabilities of a Web Site: A Theoretical Perspective , 2002 .

[37]  Linda G. Pierce,et al.  The Perceived Utility of Human and Automated Aids in a Visual Detection Task , 2002, Hum. Factors.

[38]  Janet A. Sniezek,et al.  Trust, Confidence, and Expertise in a Judge-Advisor System. , 2001, Organizational behavior and human decision processes.

[39]  Eta S. Berner,et al.  Research Paper: Influence of Case and Physician Characteristics on Perceptions of Decision Support Systems , 1999, J. Am. Medical Informatics Assoc..

[40]  Ellen Timminga,et al.  Persuasiveness of expert systems , 1998, Behav. Inf. Technol..

[41]  P. Ring.  Fragile and Resilient Trust and Their Roles in Economic Exchange , 1996 .

[42]  J. Walther,et al.  Computer-mediated communication , 2011 .

[43]  J. H. Davis,et al.  An Integrative Model Of Organizational Trust , 1995 .

[44]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[45]  D. Horton,et al.  Mass communication and para-social interaction; observations on intimacy at a distance. , 1956, Psychiatry.