Optimum Reject Options for Prototype-based Classification

We analyse optimum reject strategies for prototype-based classifiers andreal-valued rejection measures, using the distance of a data point to theclosest prototype or probabilistic counterparts. We compare reject schemes withglobal thresholds, and local thresholds for the Voronoi cells of theclassifier. For the latter, we develop a polynomial-time algorithm to computeoptimum thresholds based on a dynamic programming scheme, and we propose anintuitive linear time, memory efficient approximation thereof with competitiveaccuracy. Evaluating the performance in various benchmarks, we conclude thatlocal reject options are beneficial in particular for simple prototype-basedclassifiers, while the improvement is less pronounced for advanced models. Forthe latter, an accuracy-reject curve which is comparable to support vectormachine classifiers with state of the art reject options can be reached.

[1]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[2]  Anil K. Jain,et al.  Reject option for VQ-based Bayesian classification , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[3]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[4]  Radu Herbei,et al.  Classification with reject option , 2006 .

[5]  Martin A. Riedmiller,et al.  Incremental GRLVQ: Learning relevant features for 3D object recognition , 2008, Neurocomputing.

[6]  Alex Alves Freitas,et al.  Comprehensible classification models: a position paper , 2014, SKDD.

[7]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[8]  Nicolai Petkov,et al.  Assessment of acrosome state in boar spermatozoa heads using n-contours descriptor and RLVQ , 2013, Comput. Methods Programs Biomed..

[9]  Frank-Michael Schleif,et al.  Learning vector quantization for (dis-)similarities , 2014, Neurocomputing.

[10]  Bruno Mirbach,et al.  Confidence Estimation in Classification Decision: A Method for Detecting Unseen Patterns , 2006 .

[11]  Paulo J. G. Lisboa,et al.  Making machine learning models interpretable , 2012, ESANN.

[12]  Padraig Cunningham,et al.  Generating Estimates of Classification Confidence for a Case-Based Spam Filter , 2005, ICCBR.

[13]  Frank-Michael Schleif,et al.  Adaptive conformal semi-supervised vector quantization for dissimilarity data , 2014, Pattern Recognit. Lett..

[14]  Michael Biehl,et al.  Dynamics and Generalization Ability of LVQ Algorithms , 2007, J. Mach. Learn. Res..

[15]  Michael Biehl,et al.  Analysis of Flow Cytometry Data by Matrix Relevance Learning Vector Quantization , 2013, PloS one.

[16]  L. K. Hansen,et al.  The Error-Reject Tradeoff , 1997 .

[17]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[18]  Heiko Wersing,et al.  Rejection strategies for learning vector quantization , 2014, ESANN.

[19]  Blaise Hanczar,et al.  Accuracy-Rejection Curves (ARCs) for Comparing Classification Methods with a Reject Option , 2009, MLSB.

[20]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[21]  Mario Vento,et al.  To reject or not to reject: that is the question-an answer in case of neural classifiers , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[22]  Michael Biehl,et al.  Hyperparameter learning in probabilistic prototype-based models , 2010, Neurocomputing.

[23]  Sarah Jane Delany,et al.  Sampling with Confidence: Using k-NN Confidence Measures in Active Learning , 2009 .

[24]  Karsten M. Borgwardt,et al.  Rapid Distance-Based Outlier Detection via Sampling , 2013, NIPS.

[25]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[26]  Michael Biehl,et al.  Urine Steroid Metabolomics as a Biomarker Tool for Detecting Malignancy in Adrenal Tumors , 2011, The Journal of clinical endocrinology and metabolism.

[27]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[28]  Eiki Ishidera,et al.  A confidence value estimation method for handwritten Kanji character recognition and its application to candidate reduction , 2003, Document Analysis and Recognition.

[29]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[30]  Heiko Wersing,et al.  Local Rejection Strategies for Learning Vector Quantization , 2014, ICANN.

[31]  Haibo He,et al.  Learning and modeling big data , 2014, ESANN.

[32]  Klaus Obermayer,et al.  Dynamic Hyperparameter Scaling Method for LVQ Algorithms , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[33]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[34]  Klaus Obermayer,et al.  Soft Learning Vector Quantization , 2003, Neural Computation.

[35]  Jaime S. Cardoso,et al.  The data replication method for the classification with reject option , 2010, AI Commun..

[36]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[37]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[38]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[39]  Gyan Bhanot,et al.  Inter-species prediction of protein phosphorylation in the sbv IMPROVER species translation challenge , 2015, Bioinform..

[40]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[41]  Axel Wismüller,et al.  Texture feature ranking with relevance learning to classify interstitial lung disease patterns , 2012, Artif. Intell. Medicine.

[42]  Thomas Villmann,et al.  Rejection Strategies for Learning Vector Quantization - A Comparison of Probabilistic and Deterministic Approaches , 2014, WSOM.

[43]  Ronald L. Rivest,et al.  Introduction to Algorithms, Second Edition , 2001 .

[44]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[45]  Michael Biehl,et al.  Matrix relevance LVQ in steroid metabolomics based classification of adrenal tumors , 2012, ESANN.

[46]  Heiko Wersing,et al.  Efficient rejection strategies for prototype-based classification , 2015, Neurocomputing.

[47]  Thomas Villmann,et al.  Regularization in Matrix Relevance Learning , 2010, IEEE Transactions on Neural Networks.

[48]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[49]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[50]  Cynthia Rudin,et al.  Machine learning for science and society , 2013, Machine Learning.

[51]  Sridhar Ramaswamy,et al.  Efficient algorithms for mining outliers from large data sets , 2000, SIGMOD '00.

[52]  Fabio Roli,et al.  Reject option with multiple thresholds , 2000, Pattern Recognit..

[53]  Horst-Michael Groß,et al.  A life-long learning vector quantization approach for interactive learning of multiple categories , 2012, Neural Networks.