Criteria of Efficiency for Conformal Prediction

We study optimal conformity measures for various criteria of efficiency in an idealised setting. This leads to an important class of criteria of efficiency that we call probabilistic; it turns out that the most standard criteria of efficiency used in literature on conformal prediction are not probabilistic.

[1]  Vladimir Vovk,et al.  Comparing the Bayes and Typicalness Frameworks , 2001, ECML.

[2]  Henrik Boström,et al.  Evolved decision trees as conformal predictors , 2013, 2013 IEEE Congress on Evolutionary Computation.

[3]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[4]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[5]  Larry A. Wasserman,et al.  Least Ambiguous Set-Valued Classifiers With Bounded Error Levels , 2016, Journal of the American Statistical Association.

[6]  Alexander Gammerman,et al.  Conformal Prediction under Hypergraphical Models , 2013, AIAI.

[7]  Harris Papadopoulos,et al.  Guest editors’ preface to the special issue on conformal prediction and its applications , 2014, Annals of Mathematics and Artificial Intelligence.

[8]  J. Robins,et al.  Distribution-Free Prediction Sets , 2013, Journal of the American Statistical Association.

[9]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[10]  Vladimir Vovk,et al.  Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications , 2014 .

[11]  Jing Lei.  Classification with confidence , 2014 .

[12]  Larry Wasserman,et al.  Distribution‐free prediction bands for non‐parametric regression , 2014 .

[13]  Vladimir Vovk,et al.  From conformal to probabilistic prediction , 2014, AIAI Workshops.

[14]  Paolo Toth,et al.  Knapsack Problems: Algorithms and Computer Implementations , 1990 .

[15]  Alexander Gammerman,et al.  Anomaly Detection of Trajectories with Kernel Density Estimation by Conformal Prediction , 2014, AIAI Workshops.

[16]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.

[17]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[18]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .