Inductive Venn Prediction

Venn Predictors (VPs) are machine learning algorithms that can provide well calibrated multiprobability outputs for their predictions. An important drawback of Venn Predictors is their computational inefficiency, especially in the case of large datasets. In this work, we investigate and propose Inductive Venn Predictors (IVPs), which can overcome the computational inefficiency problem of the original Transductive Venn Prediction framework. We develop an IVP algorithm and perform a detailed comparison of its time efficiency, accuracy, and quality of probabilistic outputs with those of the original Transductive Venn Predictor (TVP). The results demonstrate that our method provides well calibrated results while maintaining high accuracy. The IVP outperforms the original TVP method in terms of time efficiency, while also providing well-calibrated probabilistic estimates. Another observation is that the probability intervals of the IVP are tighter than those of the TVP.

[1]  Harris Papadopoulos,et al.  Reliable probabilistic classification with neural networks , 2013, Neurocomputing.

[2]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[3]  Harris Papadopoulos,et al.  Evaluation of the Risk of Stroke with Confidence Predictions Based on Ultrasound Carotid Image Analysis , 2012, Int. J. Artif. Intell. Tools.

[4]  H. Cai,et al.  An Affine Invariant Region Detector Using the 4th Differential Invariant , 2007 .

[5]  A. Gammerman,et al.  Statistical Applications in Genetics and Molecular Biology , 2011 .

[6]  Ilia Nouretdinov,et al.  Prediction with Confidence Based on a Random Forest Classifier , 2010, AIAI.

[7]  Harris Papadopoulos,et al.  Reliable Confidence Intervals for Software Effort Estimation , 2009, AIAI Workshops.

[8]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[9]  Alexander Gammerman,et al.  Reliable classification of childhood acute leukaemia from gene expression data using confidence machines , 2006, 2006 IEEE International Conference on Granular Computing.

[10]  M. Bohanec,et al.  KNOWLEDGE ACQUISITION AND EXPLANATION FOR MULTI-ATTRIBUTE DECISION MAKING ∗ , 1988 .

[11]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[12]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[13]  Zhiyuan Luo,et al.  Reliable Probabilistic Classification and Its Application to Internet Traffic , 2008, ICIC.

[14]  Alexander Gammerman,et al.  Machine-Learning Applications of Algorithmic Randomness , 1999, ICML.

[15]  Alexander Gammerman,et al.  Transductive Confidence Machines for Pattern Recognition , 2002, ECML.

[16]  Alexander Gammerman,et al.  A Comparison of Venn Machine with Platt's Method in Probabilistic Outputs , 2011, EANN/AIAI.

[17]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[18]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[19]  Zhiyuan Luo,et al.  Predictions with Confidence in Applications , 2009, MLDM.

[20]  Harris Papadopoulos,et al.  Reliable Confidence Measures for Medical Diagnosis With Evolutionary Algorithms , 2011, IEEE Transactions on Information Technology in Biomedicine.

[21]  Harris Papadopoulos,et al.  Qualified Prediction for Large Data Sets in the Case of Pattern Recognition , 2002, International Conference on Machine Learning and Applications.

[22]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.

[23]  Harris Papadopoulos,et al.  Reliable Probabilistic Prediction for Medical Decision Support , 2011, EANN/AIAI.

[24]  Harris Papadopoulos,et al.  Inductive Conformal Prediction: Theory and Application to Neural Networks , 2008 .

[25]  Harris Papadopoulos,et al.  Conformal Prediction with Neural Networks , 2007, 19th IEEE International Conference on Tools with Artificial Intelligence(ICTAI 2007).

[26]  Vladimir Vovk,et al.  Self-calibrating Probability Forecasting , 2003, NIPS.

[27]  Harris Papadopoulos,et al.  Reliable Probability Estimates Based on Support Vector Machines for Large Multiclass Datasets , 2012, AIAI.

[28]  Bianca Zadrozny,et al.  Transforming classifier scores into accurate multiclass probability estimates , 2002, KDD.

[29]  Paulo Cortez,et al.  Modeling wine preferences by data mining from physicochemical properties , 2009, Decis. Support Syst..

[30]  Harris Papadopoulos,et al.  Inductive Confidence Machines for Regression , 2002, ECML.

[31]  Harris Papadopoulos,et al.  Regression Conformal Prediction with Nearest Neighbours , 2014, J. Artif. Intell. Res..

[32]  Harris Papadopoulos,et al.  Reliable diagnosis of acute abdominal pain with conformal prediction , 2009 .

[33]  Joseph Drish,et al.  Obtaining Calibrated Probability Estimates from Support Vector Machines , 2001 .