Transductive reliability estimation for medical diagnosis

In the past decades, machine learning (ML) tools have been successfully used in several medical diagnostic problems. While they often significantly outperform expert physicians (in terms of diagnostic accuracy, sensitivity, and specificity), they are mostly not being used in practice. One reason for this is that it is difficult to obtain an unbiased estimation of diagnose's reliability. We discuss how reliability of diagnoses is assessed in medical decision-making and propose a general framework for reliability estimation in machine learning, based on transductive inference. We compare our approach with a usual (machine learning) probabilistic approach as well as with classical stepwise diagnostic process where reliability of diagnose is presented as its post-test probability. The proposed transductive approach is evaluated on several medical datasets from the University of California (UCI) repository as well as on a practical problem of clinical diagnosis of the coronary artery disease (CAD). In all cases, significant improvements over existing techniques are achieved.

[1]  Kai Ming Ting,et al.  Decision Combination Based on the Characterisation of Predictive Accuracy , 1997, Intell. Data Anal..

[2]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[3]  Alexander G. Gray,et al.  Retrofitting Decision Tree Classifiers Using Kernel Density Estimation , 1995, ICML.

[4]  Martin Crowder,et al.  Statistical Analysis of Reliability Data , 1991 .

[5]  Johannes Fürnkranz,et al.  An Evaluation of Grading Classifiers , 2001, IDA.

[6]  Igor Kononenko,et al.  Reliable Classifications with Machine Learning , 2002, ECML.

[7]  G. Diamond,et al.  Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. , 1979, The New England journal of medicine.

[8]  Shlomo Argamon,et al.  Arbitrating Among Competing Classifiers Using Learned Referees , 2001, Knowledge and Information Systems.

[9]  Igor Kononenko,et al.  Semi-Naive Bayesian Classifier , 1991, EWSL.

[10]  R. Rakel,et al.  Primary Cardiology , 1999, Annals of Internal Medicine.

[11]  Matjaz Kukar.  Making Reliable Diagnoses with Machine Learning: A Case Study , 2001, AIME.

[12]  Inder Jeet Taneja,et al.  On Generalized Information Measures and Their Applications , 1989 .

[13]  M. Gerson,et al.  Cardiac Nuclear Medicine , 1996 .

[14]  A. C. Kimber,et al.  Statistical concepts in reliability , 1991 .

[15]  Richard Von Mises,et al.  Mathematical Theory of Probability and Statistics , 1966 .

[16]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[17]  J. Candell‐Riera,et al.  Nuclear Cardiology in Everyday Practice , 1994 .

[18]  I. Kononenko,et al.  Machine learning improves the accuracy of coronary artery disease diagnostic methods , 1997, Computers in Cardiology 1997.

[19]  Stephen D. Bay,et al.  Characterizing Model Errors and Di erences , 2000 .

[20]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.

[21]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 2019, Texts in Computer Science.

[22]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[23]  Gries,et al.  The chi-square test for independence , 2022 .

[24]  Igor Kononenko,et al.  Inductive and Bayesian learning in medical diagnosis , 1993, Appl. Artif. Intell..

[25]  J. Candell‐Riera,et al.  Nuclear Cardiology in Everyday Practice , 1994, Developments in Cardiovascular Medicine.

[26]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[27]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[28]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[29]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[30]  Simon French,et al.  Statistical Analysis of Reliability Data , 1992 .

[31]  M. Olona-Cabases,et al.  The probability of a correct diagnosis , 1994 .

[32]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[33]  D. D. Bono,et al.  CARDIOLOGY , 1990, The Lancet.

[34]  Alexander Gammerman,et al.  Machine-Learning Applications of Algorithmic Randomness , 1999, ICML.

[35]  Igor Kononenko,et al.  Analysing and improving the diagnosis of ischaemic heart disease with machine learning , 1999, Artif. Intell. Medicine.

[36]  O. M. Halck,et al.  Using Hard Classifiers to Estimate Conditional Class Probabilities , 2002, ECML.

[37]  R. Lyman Ott.,et al.  An introduction to statistical methods and data analysis , 1977 .

[38]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[39]  Harald Bergstriim.  Mathematical Theory of Probability and Statistics , 1966 .

[40]  Tapio Elomaa,et al.  Proceedings of the 13th European Conference on Machine Learning , 2002 .

[41]  William H. Press,et al.  Numerical recipes in C , 2002 .

[42]  Matjaz Kukar,et al.  Machine Learning in Stepwise Diagnostic Process , 1999, AIMDM.

[43]  Stephen D. Bay,et al.  Characterizing Model Erros and Differences , 2000, ICML.