The Medical Authority of AI: A Study of AI-enabled Consumer-Facing Health Technology

Recently, consumer-facing health technologies such as Artificial Intelligence (AI)-based symptom checkers (AISCs) have sprung up in everyday healthcare practice. AISCs solicit symptom information from users and provide medical suggestions and possible diagnoses, a responsibility that people usually entrust with real-person authorities such as physicians and expert patients. Thus, the advent of AISCs begs a question of whether and how they transform the notion of medical authority in people's everyday healthcare practice. To answer this question, we conducted an interview study with thirty AISC users. We found that users assess the medical authority of AISCs using various factors including AISCs’ automated decisions and interaction design patterns, associations with established medical authorities like hospitals, and comparisons with other health technologies. We reveal how AISCs are used in healthcare delivery, discuss how AI transforms conventional understandings of medical authority, and derive implications for designing AI-enabled health technology.

[1]  Jodi Forlizzi,et al.  Robot Presence and Human Honesty: Experimental Evidence , 2015, 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[2]  David J. Hutson,et al.  "Your body is your business card": Bodily capital and health authority in the fitness industry. , 2013, Social science & medicine.

[3]  J. Rowland,et al.  Information needs and sources of information among cancer patients: a systematic review of research (1980-2003). , 2005, Patient education and counseling.

[4]  B. J. Visser,et al.  Medical apps for smartphones: lack of evidence undermines quality and safety , 2012, Evidence-Based Medicine.

[5]  D. Marr,et al.  Artificial Intelligence - A Personal View , 1976, Artif. Intell..

[6]  Iris D. Tommelein,et al.  Site‐Layout Modeling: How Can Artificial Intelligence Help? , 1992 .

[7]  Paul Wicks,et al.  Patients’ Utilization and Perception of an Artificial Intelligence–Based Symptom Assessment and Advice Technology in a British Primary Care Waiting Room: Exploratory Pilot Study , 2020, JMIR human factors.

[8]  S. Nundy,et al.  Comparison of Physician and Computer Diagnostic Accuracy. , 2016, JAMA internal medicine.

[9]  Karrie Karahalios,et al.  Communicating Algorithmic Process in Online Behavioral Advertising , 2018, CHI.

[10]  Jan Flowers,et al.  Qualitative evaluation of a mobile phone and web-based collaborative care intervention for patients with type 2 diabetes. , 2011, Diabetes technology & therapeutics.

[11]  Benjamin Marshall Davies,et al.  A Novel Insight Into the Challenges of Diagnosing Degenerative Cervical Myelopathy Using Web-Based Symptom Checkers , 2019, Journal of medical Internet research.

[12]  Deborah Lupton,et al.  'It's like having a physician in your pocket!' A critical analysis of self-diagnosis smartphone apps. , 2015, Social science & medicine.

[13]  R. Pentz,et al.  Futile care in oncology: when to stop trying. , 2001, The Lancet. Oncology.

[14]  M K O'Brien,et al.  Adherence to Medication Regimens: Updating a Complex Medical Issue , 1992, Medical care review.

[15]  Paul C. Tang,et al.  Informing patients: a guide for providing patient health information. , 1998, Journal of the American Medical Informatics Association : JAMIA.

[16]  Kenneth D. Mandl,et al.  Social but safe? Quality and safety of diabetes-related online social networks , 2011, J. Am. Medical Informatics Assoc..

[17]  Rabia Kahveci,et al.  Shared Decision-Making in Pediatric Intensive Care Units: A Qualitative Study with Physicians, Nurses and Parents , 2014, The Indian Journal of Pediatrics.

[18]  Qian Yang,et al.  Designing Theory-Driven User-Centric Explainable AI , 2019, CHI.

[19]  Bo Jin,et al.  Which Doctor to Trust: A Recommender System for Identifying the Right Doctors , 2016, Journal of medical Internet research.

[20]  John Powell,et al.  Trust Me, I’m a Chatbot: How Artificial Intelligence in Health Care Fails the Turing Test , 2019, Journal of medical Internet research.

[21]  S. Zipfel,et al.  “But Dr Google said…” – Training medical students how to communicate with E-patients , 2019, Medical teacher.

[22]  Bing Wu,et al.  Advances in Online Authority Research , 2014 .

[23]  Avi Goldfarb,et al.  Exploring the Impact of Artificial Intelligence: Prediction Versus Judgment , 2018, Information Economics and Policy.

[24]  Hakan Kardes,et al.  Personalized Symptom Checker Using Medical Claims , 2018, HealthRecSys@RecSys.

[25]  Jessica A. Chen,et al.  Conversational agents in healthcare: a systematic review , 2018, J. Am. Medical Informatics Assoc..

[26]  Ewout W Steyerberg,et al.  The price of autonomy: should we offer individuals a choice of colorectal cancer screening strategies? , 2013, The Lancet. Oncology.

[27]  Jianqiang Li,et al.  Emerging information technologies for enhanced healthcare , 2015, Comput. Ind..

[28]  Xuan Huang,et al.  Assessing the authority of free online scholarly information , 2011, Scientometrics.

[29]  Yair Galily,et al.  Artificial intelligence and sports journalism: Is it a sweeping change? , 2018, Technology in Society.

[30]  Mislav Čavka,et al.  Historical and Social Evolution of the Healers’ istorical and Social Evolution of the Healers’ Charisma , 2015 .

[31]  Joonhwan Lee,et al.  Comparing Data from Chatbot and Web Surveys: Effects of Platform and Conversational Style on Survey Response Quality , 2019, CHI.

[32]  Helena M. Mentis,et al.  Turning to Peers: Integrating Understanding of the Self, the Condition, and Others’ Experiences in Making Sense of Complex Chronic Conditions , 2016, Computer Supported Cooperative Work (CSCW).

[33]  G. Eysenbach,et al.  The Impact of CyberHealthcare on the Physician–Patient Relationship , 2003, Journal of Medical Systems.

[34]  Mark S. Ackerman,et al.  Toward Health Information Technology that Supports Overweight/Obese Women in Addressing Emotion- and Stress-Related Eating , 2018, CHI.

[35]  M. Haug,et al.  The erosion of professional authority: a cross-cultural inquiry in the case of the physician. , 1976, The Milbank Memorial Fund quarterly. Health and society.

[36]  Devon L. Greyson.  Health information practices of young parents , 2017, J. Documentation.

[37]  Heinrich Hußmann,et al.  When people and algorithms meet: user-reported problems in intelligent everyday applications , 2019, IUI.

[38]  Arthur Isak Applbaum,et al.  The Idea of Legitimate Authority in the Practice of Medicine. , 2017, AMA journal of ethics.

[39]  Pamela J. Hinds,et al.  Whose Job Is It Anyway? A Study of Human-Robot Interaction in a Collaborative Task , 2004, Hum. Comput. Interact..

[40]  Corina Sas,et al.  From Biodata to Somadata , 2020, CHI.

[41]  Sean A. Munson,et al.  Boundary Negotiating Artifacts in Personal Informatics: Patient-Provider Collaboration with Patient-Generated Data , 2015, CSCW.

[42]  Amber T. Loos,et al.  Cyberchondria: Too Much Information for the Health Anxious Patient? , 2013 .

[43]  Caitlin Lustig,et al.  Algorithmic Authority: The Case of Bitcoin , 2015, 2015 48th Hawaii International Conference on System Sciences.

[44]  Johan Gustav Bellika,et al.  Combining multivariate statistics and the think-aloud protocol to assess Human-Computer Interaction barriers in symptom checkers , 2017, J. Biomed. Informatics.

[45]  Roger Burrows,et al.  The mundane realities of the everyday lay use of the internet for health, and their consequences for media convergence. , 2005, Sociology of health & illness.

[46]  Francesco Biondi,et al.  Human–Vehicle Cooperation in Automated Driving: A Multidisciplinary Review and Appraisal , 2019, Int. J. Hum. Comput. Interact..

[47]  Lisa Lundy,et al.  Women, Pregnancy, and Health Information Online , 2012 .

[48]  Milton I. Roemer,et al.  Professional Dominance: The Social Structure of Medical Care , 1972 .

[49]  Amy S Porter,et al.  Problems with the problem list: challenges of transparency in an era of patient curation , 2020, J. Am. Medical Informatics Assoc..

[50]  Laura Senier,et al.  The strategic defense of physician autonomy: State public health agencies as countervailing powers. , 2017, Social science & medicine.

[51]  A. Broom.  Virtually He@lthy: The Impact of Internet Use on Disease Experience and the Doctor-Patient Relationship , 2005, Qualitative health research.

[52]  T Marshall.  Scientific knowledge in medicine: a new clinical epistemology? , 1997, Journal of evaluation in clinical practice.

[53]  Rajesh Chandwani,et al.  Who's the Doctor?: Physicians' Perception of Internet Informed Patients in India , 2016, CHI.

[54]  S. Peters,et al.  Patients with medically unexplained symptoms: sources of patients' authority and implications for demands on medical care. , 1998, Social science & medicine.

[55]  Nick Wilson,et al.  Rapid developments in artificial intelligence: how might the New Zealand government respond? , 2017 .

[56]  Jeffrey Heer,et al.  The Effects of Interactive Latency on Exploratory Visual Analysis , 2014, IEEE Transactions on Visualization and Computer Graphics.

[57]  Federico Cabitza,et al.  Bridging the “last mile” gap between AI implementation and operation: “data awareness” that matters , 2020, Annals of translational medicine.

[58]  G. Simons,et al.  Are online symptoms checkers useful for patients with inflammatory arthritis? , 2016, BMC Musculoskeletal Disorders.

[59]  Carl Shen,et al.  Accuracy of a Popular Online Symptom Checker for Ophthalmic Diagnoses. , 2019, JAMA ophthalmology.

[60]  Keng Siau,et al.  Building Trust in Artificial Intelligence, Machine Learning, and Robotics , 2018 .

[61]  J. Mckinlay,et al.  The End of the Golden Age of Doctoring , 2002, International journal of health services : planning, administration, evaluation.

[62]  Michael A. Goodrich,et al.  Sliding Autonomy for UAV Path-Planning: Adding New Dimensions to Autonomy Management , 2015, AAMAS.

[63]  C. Gidengil,et al.  Evaluation of symptom checkers for self diagnosis and triage: audit study , 2015, BMJ : British Medical Journal.

[64]  Hussein A. Abbass,et al.  Social Integration of Artificial Intelligence: Functions, Automation Allocation Logic and Human-Autonomy Trust , 2019, Cognitive Computation.

[65]  Yura N. Perov,et al.  A comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis , 2018, ArXiv.

[66]  Alicia García-Holgado,et al.  Dashboard Meta-Model for Knowledge Management in Technological Ecosystem: A Case Study in Healthcare , 2019, UCAmI.

[67]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[68]  Susan Athey,et al.  The Allocation of Decision Authority to Human and Artificial Intelligence , 2020, AEA Papers and Proceedings.

[69]  Sierra Holland,et al.  Constructing queer mother-knowledge and negotiating medical authority in online lesbian pregnancy journals. , 2018, Sociology of health & illness.

[70]  Allison Woodruff,et al.  A Qualitative Exploration of Perceptions of Algorithmic Fairness , 2018, CHI.

[71]  Terttu Kortelainen,et al.  The cognitive authority of user-generated health information in an online forum for girls and young women , 2019, J. Documentation.

[72]  Deborah Lupton,et al.  Apps as Artefacts: Towards a Critical Perspective on Mobile Health and Medical Apps , 2014 .

[73]  Katie J. Ward,et al.  The 'expert patient': empowerment or medical dominance? The case of weight loss, pharmaceutical drugs and the Internet. , 2005, Social science & medicine.

[74]  Anders Baerheim,et al.  [Medical professionalism]. , 2007, Tidsskrift for den Norske laegeforening : tidsskrift for praktisk medicin, ny raekke.

[75]  B. Cash,et al.  Online symptom checker diagnostic and triage accuracy for HIV and hepatitis C , 2019, Epidemiology and Infection.

[76]  Rino Falcone,et al.  Socio-Cognitive Model of Trust , 2005, Encyclopedia of Information Science and Technology.

[77]  Peter Gall Krogh,et al.  Interactive Interior and Proxemics Thresholds: Empowering Participants in Sensitive Conversations , 2018, CHI.

[78]  M. Barry,et al.  Shared decision making--pinnacle of patient-centered care. , 2012, The New England journal of medicine.

[79]  Alexander Monz,et al.  Investigating Perceptions of Social Intelligence in Simulated Human-Chatbot Interactions , 2020, IIH-MSP.

[80]  Claus Bossen,et al.  Data Work in Healthcare: Challenges for Patients, Clinicians and Administrators , 2018, CSCW Companion.

[81]  Marcus Hutter,et al.  One Decade of Universal Artificial Intelligence , 2012, ArXiv.

[82]  Reijo Savolainen,et al.  Media credibility and cognitive authority. The case of seeking orienting information , 2007, Inf. Res..

[83]  Xinning Gui,et al.  Self-Diagnosis through AI-enabled Chatbot-based Symptom Checkers: User Experiences and Design Considerations , 2021, AMIA.

[84]  Wanda Pratt,et al.  Supporting cancer patients' unanchored health information management with mobile technology. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[85]  Xianghua Ding,et al.  Getting the Healthcare We Want: The Use of Online "Ask the Doctor" Platforms in Practice , 2020, CHI.

[86]  Min Kyung Lee.  Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management , 2018, Big Data Soc..

[87]  David Williams,et al.  Psychosocial Factors in Medical and Psychological Treatment Avoidance: The Role of the Doctor–Patient Relationship , 2004, Journal of health psychology.

[88]  Elisa Giaccardi,et al.  Co-performance: Conceptualizing the Role of Artificial Agency in the Design of Everyday Life , 2018, CHI.

[89]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[90]  Hardeep Singh,et al.  Patient Perspectives on the Usefulness of an Artificial Intelligence–Assisted Symptom Checker: Cross-Sectional Survey Study , 2020, Journal of medical Internet research.

[91]  M. Hardey.  Doctor in the house: the Internet as a source of lay health knowledge and the challenge to expertise , 1999 .

[92]  Aaron Halfaker,et al.  Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems , 2020, CHI.

[93]  Paul Farber,et al.  Authority in Education , 1986, Teachers College Record: The Voice of Scholarship in Education.

[94]  Christian Sandvig,et al.  Algorithmic Authority: the Ethics, Politics, and Economics of Algorithms that Interpret, Decide, and Manage , 2016, CHI Extended Abstracts.

[95]  T. Greenhalgh.  Patient and public involvement in chronic illness: beyond the expert patient , 2009, BMJ : British Medical Journal.

[96]  J. Powell,et al.  ‘Oh dear, should I really be saying that on here?’: Issues of identity and authority in an online diabetes community , 2010, Health.

[97]  Ruth Thephila Samson Rajakumar.  Comparison of Usability of PC Based and Smartphone Symptom Checker , 2019 .

[98]  Guy H. Walker,et al.  Where Is Computing Driving Cars? , 2001, Int. J. Hum. Comput. Interact..

[99]  W. Bradley Wendel.  The Promise and Limitations of Artificial Intelligence in the Practice of Law , 2019 .

[100]  Lieke Van Deinsen,et al.  Found through Translation: Female Translators and the Construction of ‘Relational Authority’ in the Eighteenth-Century Dutch Republic , 2019, Early Modern Low Countries.

[101]  Christopher Pearce,et al.  The patient and the computer in the primary care consultation , 2011, J. Am. Medical Informatics Assoc..

[102]  F. Sardanelli,et al.  Artificial intelligence in medical imaging: threat or opportunity? Radiologists again at the forefront of innovation in medicine , 2018, European Radiology Experimental.

[103]  Enrico Coiera,et al.  Safety of patient-facing digital symptom checkers , 2018, The Lancet.

[104]  Margaret McCartney,et al.  Margaret McCartney: AI in medicine must be rigorously tested , 2018, British Medical Journal.

[105]  Alka V Menon,et al.  Do online reviews diminish physician authority? The case of cosmetic surgery in the U.S. , 2017, Social science & medicine.

[106]  Elie A. Akl,et al.  Factors associated with the public’s trust in physicians in the context of the Lebanese healthcare system: a qualitative study , 2019, BMC Health Services Research.

[107]  Joo-Wha Hong,et al.  Bias in Perception of Art Produced by Artificial Intelligence , 2018, HCI.

[108]  Mark O. Riedl,et al.  Automated rationale generation: a technique for explainable AI and its effects on human perceptions , 2019, IUI.

[109]  Ali Parsa,et al.  Sorting out symptoms: design and evaluation of the 'babylon check' automated triage system , 2016, ArXiv.

[110]  Soo Young Rieh.  Judgment of information quality and cognitive authority in the Web , 2002, J. Assoc. Inf. Sci. Technol..

[111]  Enrico Coiera,et al.  The Last Mile: Where Artificial Intelligence Meets Reality , 2019, Journal of medical Internet research.

[112]  William A. Kahn,et al.  Authority at Work: Internal Models and Their Organizational Consequences , 1994 .

[113]  Paul N. Bennett,et al.  Guidelines for Human-AI Interaction , 2019, CHI.

[114]  Souraya Sidani,et al.  Effects of patient-centered care on patient outcomes: an evaluation. , 2008, Research and theory for nursing practice.

[115]  Charlene Liew.  The future of radiology augmented with Artificial Intelligence: A strategy for success. , 2018, European journal of radiology.

[116]  R. Weitz,et al.  Physician satisfaction in a changing health care environment: the impact of challenges to professional autonomy, authority, and dominance. , 1998, Journal of health and social behavior.

[117]  Rebecca Schaffer,et al.  Producing genetic knowledge and citizenship through the Internet: mothers, pediatric genetics, and cybermedicine. , 2008, Sociology of health & illness.

[118]  Anssi Peräkylä,et al.  Authority and Accountability: The Delivery of Diagnosis in Primary Health Care , 1998 .

[119]  Cristiano André da Costa,et al.  Survey of conversational agents in health , 2019, Expert Syst. Appl..

[120]  Michelle X. Zhou,et al.  If I Hear You Correctly: Building and Evaluating Interview Chatbots with Active Listening Skills , 2020, CHI.