Precision-Recall-Optimization in Learning Vector Quantization Classifiers for Improved Medical Classification Systems

Classification and decision systems in data analysis are mostly based on accuracy optimization. This criterion is only a conditional informative value if the data are imbalanced or false positive/negative decisions cause different costs. Therefore more sophisticated statistical quality measures are favored in medicine, like precision, recall etc.. Otherwise, most classification approaches in machine learning are designed for accuracy optimization. In this paper we consider variants of learning vector quantizers (LVQs) explicitly optimizing those advanced statistical quality measures while keeping the basic intuitive ingredients of these classifiers, which are the prototype based principle and the Hebbian learning. In particular we focus in this contribution particularly to precision and recall as important measures for use in medical applications. We investigate these problems in terms of precision-recall curves as well as receiver-operating characteristic (ROC) curves well-known in statistical classification and test analysis. With the underlying more general framework, we provide a principled alternatives traditional classifiers, such that a closer connection to statistical classification analysis can be drawn.

[1]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[2]  Shehroz S. Khan,et al.  A Survey of Recent Trends in One Class Classification , 2009, AICS.

[3]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[4]  Thomas Villmann,et al.  Aspects in Classification Learning - Review of Recent Developments in Learning Vector Quantization , 2014 .

[5]  Thomas Villmann,et al.  Border-Sensitive Learning in Kernelized Learning Vector Quantization , 2013, IWANN.

[6]  Ulf Brefeld,et al.  {AUC} maximizing support vector learning , 2005 .

[7]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[8]  Michael Biehl,et al.  ROC-Optimization and Statistical Quality Measures in Learning Vector Quantization Classifiers , 2014 .

[9]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[10]  Thomas Villmann,et al.  FAEP und Morphometrie des Mesenzephalons bei Morbus Wilson , 2007 .

[11]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[12]  Michael Biehl,et al.  Matrix relevance LVQ in steroid metabolomics based classification of adrenal tumors , 2012, ESANN.

[13]  Thomas Villmann,et al.  Genotype Correlation with Fine Motor Symptoms in Patients with Wilson’s Disease , 2002, European Neurology.

[14]  R. A. McGill,et al.  Probabilistic Neural Networks for Chemical Sensor Array Pattern Recognition: Comparison Studies, Improvements and Automated Outlier Rejection , 1998 .

[15]  Thomas Villmann,et al.  Kernelized vector quantization in gradient-descent learning , 2015, Neurocomputing.

[16]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[17]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[18]  David Page,et al.  Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals , 2013, ECML/PKDD.

[19]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[20]  Thomas Villmann,et al.  Median Variants of LVQ for Optimization of Statistical Quality Measures for Classi cation of Dissimilarity Data , 2014 .

[21]  Eyke Hüllermeier,et al.  An Exact Algorithm for F-Measure Maximization , 2011, NIPS.

[22]  Thomas Villmann,et al.  Statistical Quality Measures and ROC-Optimization by Learning Vector Quantization Classifiers , 2014 .

[23]  Robert P. W. Duin,et al.  Precision-recall operating characteristic (P-ROC) curves in imprecise environments , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[24]  H. Altay Güvenir,et al.  Ranking Instances by Maximizing the Area under ROC Curve , 2013, IEEE Transactions on Knowledge and Data Engineering.

[25]  Michael Biehl,et al.  Analysis of Flow Cytometry Data by Matrix Relevance Learning Vector Quantization , 2013, PloS one.

[26]  Thomas Villmann,et al.  Investigation of fine-motor disturbances in Wilson's disease , 2003, Neurological Sciences.

[27]  Charles X. Ling,et al.  Using AUC and accuracy in evaluating learning algorithms , 2005, IEEE Transactions on Knowledge and Data Engineering.

[28]  Thomas Villmann,et al.  Optimization of General Statistical Accuracy Measures for Classification Based on Learning Vector Quantization , 2014, ESANN.

[29]  T. Villmann,et al.  Comparison of clinical types of Wilson's disease and glucose metabolism in extrapyramidal motor brain regions , 2002, Journal of Neurology.

[30]  Szymon Jaroszewicz,et al.  Efficient AUC Optimization for Classification , 2007, PKDD.

[31]  Salvador España Boquera,et al.  F-Measure as the Error Function to Train Neural Networks , 2013, IWANN.

[32]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[33]  Robert P. W. Duin,et al.  The interaction between classification and reject performance for distance-based reject-option classifiers , 2006, Pattern Recognit. Lett..

[34]  Thomas Villmann,et al.  Distance Measures for Prototype Based Classification , 2013, BrainComp.

[35]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[36]  Michael C. Mozer,et al.  Optimizing Classifier Performance via an Approximation to the Wilcoxon-Mann-Whitney Statistic , 2003, ICML.

[37]  J. Keilwagen,et al.  Area under Precision-Recall Curves for Weighted and Unweighted Data , 2014, PloS one.

[38]  William Nick Street,et al.  Learning to Rank by Maximizing AUC with Linear Programming , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[39]  Haibo He,et al.  Learning from Imbalanced Data , 2009, IEEE Transactions on Knowledge and Data Engineering.

[40]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[41]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[42]  T. Villmann,et al.  Elektrophysiologisches Schädigungsprofil von Patienten mit einem Morbus Wilson , 2003, Der Nervenarzt.

[43]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[44]  Filip Radlinski,et al.  A support vector method for optimizing average precision , 2007, SIGIR.

[45]  Bhavani Raskutti,et al.  Optimising area under the ROC curve using gradient descent , 2004, ICML.

[46]  Alain Rakotomamonjy,et al.  Optimizing Area Under Roc Curve with SVMs , 2004, ROCAI.

[47]  Radu Herbei,et al.  Classification with reject option , 2006 .

[48]  David Page,et al.  Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals , 2013, ECML/PKDD.

[49]  Thomas Villmann,et al.  A Median Variant of Generalized Learning Vector Quantization , 2013, ICONIP.