Discovering interobserver variability in the cytodiagnosis of breast cancer using decision trees and Bayesian networks

We evaluate the performance of two decision tree procedures and four Bayesian network classifiers as potential decision support systems in the cytodiagnosis of breast cancer. In order to test their performance thoroughly, we use two real-world databases containing 692 cases and 322 cases collected by a single observer and 19 observers, respectively. The results show that, in general, there are considerable differences in all tests (accuracy, sensitivity, specificity, PV+, PV- and ROC) when a specific classifier uses the single-observer dataset compared to those when this same classifier uses the multiple-observer dataset. These results suggest that different observers see different things: a problem known as interobserver variability. We graphically unveil such a problem by presenting the structures of the decision trees and Bayesian networks resultant from running both databases.

[1]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[2]  Joanna Cielecka-Kuszyk,et al.  Inter-observer variability in histopathological assessment of liver biopsies taken in a pediatric open label therapeutic program for chronic HBV infection treatment. , 2006, World journal of gastroenterology.

[3]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[4]  Karl Rihaczek,et al.  1. WHAT IS DATA MINING? , 2019, Data Mining for the Social Sciences.

[5]  Margaret H. Dunham,et al.  Data Mining: Introductory and Advanced Topics , 2002 .

[6]  Vojislav Kecman,et al.  Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models , 2001 .

[7]  W J Frable,et al.  Interobserver variability in the classification of proliferative breast lesions by fine‐needle aspiration: Results of the Papanicolaou Society of Cytopathology study , 1998, Diagnostic cytopathology.

[8]  R Holland,et al.  Interobserver variability between general and expert pathologists during the histopathological assessment of large-core needle and open biopsies of non-palpable breast lesions. , 2003, European journal of cancer.

[9]  H. Bozdogan,et al.  Akaike's Information Criterion and Recent Developments in Information Complexity. , 2000, Journal of mathematical psychology.

[10]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[11]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[12]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[13]  David G. Stork,et al.  Pattern Classification , 1973 .

[14]  Nada Lavrac,et al.  Selected techniques for data mining in medicine , 1999, Artif. Intell. Medicine.

[15]  Merideth Leigh Gattis,et al.  Spatial schemas and abstract thought , 2001 .

[16]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[17]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[18]  Huiman X. Barnhart,et al.  Observer Variability: A New Approach in Evaluating Interobserver Agreement , 2021, Journal of Data Science.

[19]  E John,et al.  A Process Model of Human Transitive Inference , 2001 .

[20]  Ethem Alpaydin,et al.  Introduction to Machine Learning (Adaptive Computation and Machine Learning) , 2004 .

[21]  Mark A. Pitt,et al.  Advances in Minimum Description Length: Theory and Applications , 2005 .

[22]  Lakhmi C. Jain,et al.  Artificial Intelligence Techniques in Breast Cancer Diagnosis and Prognosis , 2000 .

[23]  Gregory F. Cooper,et al.  An overview of the representation and discovery of causal relationships using Bayesian networks , 1999 .

[24]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[25]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[26]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[27]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[28]  Nir Friedman,et al.  Learning Bayesian Networks from Data , 2001 .

[29]  Ian Witten,et al.  Data Mining , 2000 .

[30]  L. M. Roberts.  MammoNet: a Bayesian Network Diagnosing Breast Cancer , 2000 .

[31]  Grünwald,et al.  Model Selection Based on Minimum Description Length. , 2000, Journal of mathematical psychology.

[32]  Lajos Diósi.  A Short Course in Quantum Information Theory , 2011 .

[33]  N. Hari Narayanan,et al.  Diagrammatic Reasoning: Cognitive and Computational Perspectives , 1995 .

[34]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[35]  Illtyd Trethowan.  Causality , 1938 .

[36]  S S Cross,et al.  Evaluation of a statistically derived decision tree for the cytodiagnosis of fine needle aspirates of the breast (FNAB) , 1998, Cytopathology : official journal of the British Society for Clinical Cytology.

[37]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[38]  Robert F. Harrison,et al.  Which Decision Support Technologies are Appropriate for the Cytodiagnosis of Breast Cancer , 2000 .

[39]  Vipin Kumar,et al.  Introduction to Data Mining , 2022, Data Mining and Machine Learning Applications.

[40]  U. Chatterjee,et al.  Effect of unconventional feeds on production cost, growth performance and expression of quantitative genes in growing pigs , 2022, Journal of the Indonesian Tropical Animal Agriculture.

[41]  Barbara Tversky,et al.  Spatial schemas in depictions , 1999 .

[42]  Andrew J Walker,et al.  Visualisation of biomedical datasets by use of growing cell structure networks: a novel diagnostic classification technique , 1999, The Lancet.

[43]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[44]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[45]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[46]  Herbert A. Simon,et al.  Why a diagram is (sometimes) worth 10, 000 word , 1987 .

[47]  T. Stephenson,et al.  Validation of a decision support system for the cytodiagnosis of fine needle aspirates of the breast using a prospectively collected dataset from multiple observers in a working clinical environment , 2000, Cytopathology : official journal of the British Society for Clinical Cytology.

[48]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[49]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .