Calibration of Machine Learning Models

One of the main goals of machine learning methods is to build a model or hypothesis from a set of data (also called evidence). After this learning process, the quality of the hypothesis must be evaluated as precisely as possible. For instance, if prediction errors have negative consequences in a certain application domain of a model (for example, detection of carcinogenic cells), it is important to know the exact ABStrAct

[1]  Elizabeth A. Peck,et al.  Introduction to Linear Regression Analysis , 2001 .

[2]  A. H. Murphy,et al.  Scalar and Vector Partitions of the Probability Score: Part I. Two-State Situation , 1972 .

[3]  H. D. Brunk,et al.  AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION , 1955 .

[4]  Peter A. Flach,et al.  Repairing Concavities in ROC Curves , 2005, IJCAI.

[5]  Peter A. Flach,et al.  A Simple Lexicographic Ranker and Probability Estimator , 2007, ECML.

[6]  Tom Fawcett,et al.  PAV and the ROC convex hull , 2007, Machine Learning.

[7]  Padraig Cunningham,et al.  Making Good Probability Estimates for Regression , 2006, ECML.

[8]  José Hernández-Orallo,et al.  An experimental comparison of performance measures for classification , 2009, Pattern Recognit. Lett..

[9]  Peter A. Flach,et al.  Improving Accuracy and Cost of Two-class and Multi-class Probabilistic Classifiers Using ROC Curves , 2003, ICML.

[10]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[11]  Bianca Zadrozny,et al.  Transforming classifier scores into accurate multiclass probability estimates , 2002, KDD.

[12]  Mehrtash Harandi,et al.  Machine learning applications in computer vision , 2012 .

[13]  Kenji Sugawara,et al.  Sitting Posture Recognition and Location Estimation for Human-Aware Environment , 2011, Int. J. Softw. Sci. Comput. Intell..

[14]  Maya R. Gupta,et al.  Cost-sensitive multi-class classification from probability estimates , 2008, ICML '08.

[15]  Yingxu Wang.  Breakthroughs in Software Science and Computational Intelligence , 2012 .

[16]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[17]  Xenia Naidenova,et al.  Integration of the Image and NL-text Analysis/Synthesis Systems , 2012 .

[18]  Bojan Cestnik,et al.  Estimating Probabilities: A Crucial Task in Machine Learning , 1990, ECAI.

[19]  I. Good.  Corroboration, Explanation, Evolving Probability, Simplicity and a Sharpened Razor , 1968, The British Journal for the Philosophy of Science.

[20]  Marcelino Martínez-Sober,et al.  Intelligent Data Analysis for Real-Life Applications: Theory and Practice , 2012 .

[21]  Witold Pedrycz,et al.  Perspectives on Cognitive Computing and Applications , 2010, Int. J. Softw. Sci. Comput. Intell..

[22]  Masato Okada,et al.  A Cognitive Engineering Design of a Smart Phone Information Appliance with Improved Usability , 2014, Int. J. Softw. Sci. Comput. Intell..

[23]  Stephen E. Fienberg,et al.  The Comparison and Evaluation of Forecasters. , 1983 .

[24]  Rich Caruana,et al.  Data mining in metric space: an empirical analysis of supervised learning performance criteria , 2004, ROCAI.

[25]  Jinbo Bi,et al.  Regression Error Characteristic Curves , 2003, ICML.

[26]  James F. Peters,et al.  Ethology-Based Approximate Adaptive Learning: A Near Set Approach , 2009 .

[27]  Bianca Zadrozny,et al.  Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers , 2001, ICML.

[28]  Yunlong Wang,et al.  Generic Cabling of Intelligent Buildings Based on Ant Colony Algorithm , 2011, Int. J. Softw. Sci. Comput. Intell..

[29]  Carlos Ramírez,et al.  A general knowledge representation model for the acquisition of skills and concepts , 2009, 2009 8th IEEE International Conference on Cognitive Informatics.

[30]  Jacek M. Zurada,et al.  Efficiency and Scalability Methods for Computational Intellect , 2013 .

[31]  Brian M. Lucey,et al.  Making Density Forecasting Models Statistically Consistent , 2006 .

[32]  Marco Carvalho,et al.  A Scalable Approach to Network Traffic Classification for Computer Network Defense using Parallel Neural Network Classifier Architectures , 2013 .

[33]  Siddhivinayak Kulkarni,et al.  Machine Learning Algorithms for Problem Solving in Computational Applications: Intelligent Techniques , 2012 .

[34]  F. Sanders.  On Subjective Probability Forecasting , 1963 .