Guest editors’ preface to the special issue on conformal prediction and its applications

Quantifying the uncertainty of predictions produced by classification and regression techniques is an important problem in the field of Machine Learning. This special issue is dedicated to Conformal Prediction (CP), which is a recently developed framework for producing provably valid measures of confidence in predictions. It can be used for extending conventional Machine Learning algorithms and thus developing methods, called Conformal Predictors (CPs), whose predictions are guaranteed to satisfy a given level of confidence without requiring anything more than that the data are generated independently by the same probability distribution (i.i.d.). More specifically, CPs produce as their predictions prediction regions, which are sets of labels sufficient to satisfy the required level of confidence. The idea behind Conformal Prediction originated in a series of discussions at Royal Holloway, University of London, in the summer of 1996, between Alexander Gammerman, Vladimir N. Vapnik and Vladimir Vovk. These discussions, which were mainly concerned with Vapnik’s work on Support Vector Machines (SVM), led to the realization that the number of support vectors used by an SVM could serve as basis for the production of confidence measures for individual predictions (in fact, similar ideas may be traced back to the joint work of Chervonenkis and Vapnik in the 1960s: see [5], footnote 4). This initial idea was described in [8] and was later improved to its current form in [37] and [33]. The first Conformal Predictor proposed in [37] and [33], which was then called Transductive Confidence

[1]  VN Balasubramanian,et al.  Support vector machine based conformal predictors for risk of complications following a coronary Drug Eluting Stent procedure , 2009, 2009 36th Annual Computers in Cardiology Conference (CinC).

[2]  Louis Vuurpijl,et al.  Reliable Gesture Recognition with Transductive Confidence Machines , 2012 .

[3]  Vladimir Vovk,et al.  Comparing the Bayes and Typicalness Frameworks , 2001, ECML.

[4]  Alexander Gammerman,et al.  Hedging Predictions in Machine Learning: The Second Computer Journal Lecture , 2006, Comput. J..

[5]  Harris Papadopoulos,et al.  Reliable probabilistic classification with neural networks , 2013, Neurocomputing.

[6]  Göran Falkman,et al.  Online Detection of Anomalous Sub-trajectories: A Sliding Window Approach Based on Conformal Anomaly Detection and Local Outlier Factor , 2012, AIAI.

[7]  Alexey Chervonenkis Chervonenkis’s Recollections , 2015 .

[8]  Harry Wechsler,et al.  Transductive confidence machine for active learning , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[9]  Harris Papadopoulos,et al.  Osteoporosis Risk Assessment with Well-Calibrated Probabilistic Outputs , 2013, AIAI.

[10]  Harris Papadopoulos,et al.  Evaluation of the Risk of Stroke with Confidence Predictions Based on Ultrasound Carotid Image Analysis , 2012, Int. J. Artif. Intell. Tools.

[11]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[12]  Alexander Gammerman,et al.  Serum Proteomic Abnormality Predating Screen Detection of Ovarian Cancer , 2009, Comput. J..

[13]  Harris Papadopoulos,et al.  Probabilistic Prediction for the Detection of Vesicoureteral Reflux , 2013, EANN.

[14]  Harris Papadopoulos Qualified Predictions for Large Data Sets , 2009, Knowl. Eng. Rev..

[15]  Vladimir Vovk,et al.  Self-calibrating Probability Forecasting , 2003, NIPS.

[16]  Zhiyuan Luo,et al.  Reliable Probabilistic Classification and Its Application to Internet Traffic , 2008, ICIC.

[17]  Haris Haralambous,et al.  Reliable prediction intervals with regression neural networks , 2011, Neural Networks.

[18]  Alexander Gammerman,et al.  Qualified predictions for microarray and proteomics pattern diagnostics with confidence machines , 2005, Int. J. Neural Syst..

[19]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.

[20]  Harry Wechsler,et al.  A Martingale Framework for Detecting Changes in Data Streams by Testing Exchangeability , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Harris Papadopoulos,et al.  Reliable Probabilistic Prediction for Medical Decision Support , 2011, EANN/AIAI.

[22]  Vladimir Vovk,et al.  Multiprobabilistic Venn Predictors with Logistic Regression , 2012, AIAI.

[23]  Siddhartha Bhattacharyya Confidence in predictions from random tree ensembles , 2011, 2011 IEEE 11th International Conference on Data Mining.

[24]  Alexander Gammerman,et al.  Pattern Recognition and Density Estimation under the General i.i.d. Assumption , 2001, COLT/EuroCOLT.

[25]  Alexander Gammerman,et al.  A Comparison of Venn Machine with Platt's Method in Probabilistic Outputs , 2011, EANN/AIAI.

[26]  Harris Papadopoulos,et al.  Inductive Conformal Prediction: Theory and Application to Neural Networks , 2008 .

[27]  Harris Papadopoulos,et al.  Reliable Probability Estimates Based on Support Vector Machines for Large Multiclass Datasets , 2012, AIAI.

[28]  Zhiyuan Luo,et al.  Network Traffic Demand Prediction with Confidence , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[29]  Harris Papadopoulos,et al.  Reliable Confidence Measures for Medical Diagnosis With Evolutionary Algorithms , 2011, IEEE Transactions on Information Technology in Biomedicine.

[30]  Vladimir Vovk,et al.  On-line confidence machines are well-calibrated , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[31]  A. J. Gammerman,et al.  Plant promoter prediction with confidence estimation , 2005, Nucleic acids research.

[32]  Harris Papadopoulos,et al.  Inductive Confidence Machines for Regression , 2002, ECML.

[33]  Alexander Gammerman,et al.  Transductive Confidence Machines for Pattern Recognition , 2002, ECML.

[34]  Alexander Gammerman,et al.  Strangeness Minimisation Feature Selection with Confidence Machines , 2006, IDEAL.

[35]  Harris Papadopoulos,et al.  Regression Conformal Prediction with Nearest Neighbours , 2014, J. Artif. Intell. Res..

[36]  Harris Papadopoulos,et al.  Reliable diagnosis of acute abdominal pain with conformal prediction , 2009 .

[37]  Jin Zhang,et al.  Recognition of hypoxia EEG with a preset confidence level based on EEG analysis , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[38]  Alexander Gammerman,et al.  Machine-Learning Applications of Algorithmic Randomness , 1999, ICML.

[39]  Vladimir Vovk,et al.  Ridge Regression Confidence Machine , 2001, International Conference on Machine Learning.

[40]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[41]  Harris Papadopoulos,et al.  Reliable Confidence Intervals for Software Effort Estimation , 2009, AIAI Workshops.