Concordance as evidence in the Watson for Oncology decision-support system

Machine learning platforms have emerged as a new promissory technology that some argue will revolutionize work practices across a broad range of professions, including medical care. During the past few years, IBM has been testing its Watson for Oncology platform at several oncology departments around the world. Published reports, news stories, as well as our own empirical research show that in some cases, the levels of concordance over recommended treatment protocols between the platform and human oncologists have been quite low. Other studies supported by IBM claim concordance rates as high as 96%. We use the Watson for Oncology case to examine the practice of using concordance levels between tumor boards and a machine learning decision-support system as a form of evidence. We address a challenge related to the epistemic authority between oncologists on tumor boards and the Watson Oncology platform by arguing that the use of concordance levels as a form of evidence of quality or trustworthiness is problematic. Although the platform provides links to the literature from which it draws its conclusion, it obfuscates the scoring criteria that it uses to value some studies over others. In other words, the platform “black boxes” the values that are coded into its scoring system.

[1]  Steve T. Mckinlay.  Evidence, Explanation and Predictive Data Modelling , 2017 .

[2]  Alex Pentland,et al.  Fair, Transparent, and Accountable Algorithmic Decision-making Processes , 2017, Philosophy & Technology.

[3]  Tanveer Syeda-Mahmood,et al.  Role of Big Data and Machine Learning in Diagnostic Decision Support in Radiology. , 2018, Journal of the American College of Radiology : JACR.

[4]  B. Mittelstadt.  Auditing for Transparency in Content Personalization Systems , 2016 .

[5]  R. McDougall.  No we shouldn’t be afraid of medical AI; it involves risks and opportunities , 2019, Journal of Medical Ethics.

[6]  Tamar Sharon,et al.  The Googlization of health research: from disruptive innovation to disruptive ethics. , 2016, Personalized medicine.

[7]  J. Sterne,et al.  Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis , 2019, BMJ.

[8]  Tal Z. Zarsky,et al.  The Trouble with Algorithmic Decisions , 2016 .

[9]  Bo Hyun Kim,et al.  Concordance in postsurgical radioactive iodine therapy recommendations between Watson for Oncology and clinical practice in patients with differentiated thyroid carcinoma , 2019, Cancer.

[10]  R. McDougall.  Computer knows best? The need for value-flexibility in medical AI , 2018, Journal of Medical Ethics.

[11]  A. Tupasela,et al.  The Nordic data imaginary , 2020, Big Data Soc..

[12]  James H Thrall,et al.  Artificial Intelligence and Machine Learning in Radiology: Opportunities, Challenges, Pitfalls, and Criteria for Success. , 2018, Journal of the American College of Radiology : JACR.

[13]  David Schneider.  Trading at the speed of light , 2011 .

[14]  E H Shortliffe,et al.  Watson for Oncology and breast cancer treatment recommendations: agreement with an expert multidisciplinary tumor board , 2018, Annals of oncology : official journal of the European Society for Medical Oncology.

[15]  Paul K Hodgkin,et al.  The computer may be assessing you now, but who decided its values? , 2016, British Medical Journal.

[16]  Mark Buchanan,et al.  Physics in finance: Trading at the speed of light , 2015, Nature.

[17]  A. Norden,et al.  Early experience with IBM Watson for Oncology (WFO) cognitive computing system for lung and colorectal cancer treatment. , 2017 .

[18]  Young Saing Kim,et al.  Concordance Rate between Clinicians and Watson for Oncology among Patients with Advanced Gastric Cancer: Early, Real-World Experience in Korea , 2019, Canadian journal of gastroenterology & hepatology.

[19]  D. Prabhakaran,et al.  Conduct of clinical trials in developing countries: a perspective , 2009, Current opinion in cardiology.

[20]  A. Kumar,et al.  551PD Validation study to assess performance of IBM cognitive computing system Watson for oncology with Manipal multidisciplinary tumour board for 1000 consecutive cases: An Indian experience , 2016 .

[21]  Matt Carlson,et al.  Automating judgment? Algorithmic judgment, news knowledge, and journalistic professionalism , 2018, New Media Soc..

[22]  Marcello D’Agostino,et al.  Introduction: the Governance of Algorithms , 2018, Philosophy & Technology.

[23]  N. Shah,et al.  Implementing Machine Learning in Health Care - Addressing Ethical Challenges. , 2018, The New England journal of medicine.

[24]  E. Di Nucci.  Should we be afraid of medical AI? , 2019, Journal of Medical Ethics.

[25]  Pär Sparén,et al.  Occupation and cancer – follow-up of 15 million people in five Nordic countries , 2009, Acta oncologica.

[26]  M. Piccart,et al.  Keeping faith with trial volunteers , 2007, Nature.

[27]  Jie Xu,et al.  The practical implementation of artificial intelligence technologies in medicine , 2019, Nature Medicine.

[28]  H. Storm,et al.  Nordic Cancer Registries – an overview of their procedures and data comparability , 2017, Acta oncologica.

[29]  Florian Jaton,et al.  We get the algorithms of our ground truths: Designing referential databases in digital image processing , 2017, Social studies of science.

[30]  S. Tamang,et al.  Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data , 2018, JAMA internal medicine.

[31]  Chunhong Hu,et al.  Using Artificial Intelligence (Watson for Oncology) for Treatment Recommendations Amongst Chinese Patients with Lung Cancer: Feasibility Study , 2018, Journal of medical Internet research.

[32]  A. Rosoff.  THE GOLD STANDARD: THE CHALLENGE OF EVIDENCE-BASED MEDICINE AND STANDARDIZATION IN HEALTH CARE , 2004 .

[33]  Won-Suk Lee,et al.  Assessing Concordance With Watson for Oncology, a Cognitive Computing Decision Support System for Colon Cancer Treatment in Korea. , 2018, JCO clinical cancer informatics.

[34]  Do-Hoon Kim,et al.  A comparative study of Watson for Oncology and tumor boards in breast cancer treatment , 2019, Korean Journal of Clinical Oncology.

[35]  Ted Striphas.  Algorithmic culture , 2015 .

[36]  Geoffrey E. Hinton.  Deep Learning-A Technology With the Potential to Transform Health Care. , 2018, JAMA.

[37]  Judy Wajcman,et al.  Automation: is it really different this time? , 2017, The British journal of sociology.

[38]  E. Shortliffe,et al.  Artificial Intelligence Treatment Decision Support For Complex Breast Cancer Among Oncologists With Varying Expertise. , 2019, JCO clinical cancer informatics.