Reliable probabilistic classification with neural networks

Venn Prediction (VP) is a new machine learning framework for producing well-calibrated probabilistic predictions. In particular it provides well-calibrated lower and upper bounds for the conditional probability of an example belonging to each possible class of the problem at hand. This paper proposes five VP methods based on Neural Networks (NNs), which is one of the most widely used machine learning techniques. The proposed methods are evaluated experimentally on four benchmark datasets and the obtained results demonstrate the empirical well-calibratedness of their outputs and their superiority over the outputs of the traditional NN classifier.

[1]  Siddhartha Bhattacharyya,et al.  Confidence in predictions from random tree ensembles , 2011, 2011 IEEE 11th International Conference on Data Mining.

[2]  Alexander Gammerman,et al.  Transductive Confidence Machines for Pattern Recognition , 2002, ECML.

[3]  Harry Wechsler,et al.  Transductive confidence machine for active learning , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[4]  S IliadisLazaros,et al.  An Artificial Neural Network model for mountainous water-resources management , 2007 .

[5]  Alexander Gammerman,et al.  Serum Proteomic Abnormality Predating Screen Detection of Ovarian Cancer , 2009, Comput. J..

[6]  Lazaros S. Iliadis,et al.  Application of fuzzy T-norms towards a new Artificial Neural Networks' evaluation framework: A case from wood industry , 2008, Inf. Sci..

[7]  Jin Zhang,et al.  Recognition of hypoxia EEG with a preset confidence level based on EEG analysis , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[8]  Zhiyuan Luo,et al.  Reliable Probabilistic Classification and Its Application to Internet Traffic , 2008, ICIC.

[9]  Alexander Gammerman,et al.  Qualified predictions for microarray and proteomics pattern diagnostics with confidence machines , 2005, Int. J. Neural Syst..

[10]  Harry Wechsler,et al.  A Martingale Framework for Detecting Changes in Data Streams by Testing Exchangeability , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Harris Papadopoulos,et al.  Reliable Probabilistic Prediction for Medical Decision Support , 2011, EANN/AIAI.

[12]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[13]  Harris Papadopoulos,et al.  Reliable Confidence Intervals for Software Effort Estimation , 2009, AIAI Workshops.

[14]  Harris Papadopoulos,et al.  Inductive Conformal Prediction: Theory and Application to Neural Networks , 2008 .

[15]  Alexander Gammerman,et al.  A Comparison of Venn Machine with Platt's Method in Probabilistic Outputs , 2011, EANN/AIAI.

[16]  Harris Papadopoulos,et al.  Reliable Confidence Measures for Medical Diagnosis With Evolutionary Algorithms , 2011, IEEE Transactions on Information Technology in Biomedicine.

[17]  A. J. Gammerman,et al.  Plant promoter prediction with confidence estimation , 2005, Nucleic acids research.

[18]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.

[19]  Harris Papadopoulos,et al.  Inductive Confidence Machines for Regression , 2002, ECML.

[20]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[21]  VN Balasubramanian,et al.  Support vector machine based conformal predictors for risk of complications following a coronary Drug Eluting Stent procedure , 2009, 2009 36th Annual Computers in Cardiology Conference (CinC).

[22]  Louis Vuurpijl,et al.  Reliable Gesture Recognition with Transductive Confidence Machines , 2012 .

[23]  Harris Papadopoulos,et al.  Regression Conformal Prediction with Nearest Neighbours , 2014, J. Artif. Intell. Res..

[24]  Harris Papadopoulos,et al.  Reliable diagnosis of acute abdominal pain with conformal prediction , 2009 .

[25]  Vladimir Vovk,et al.  Ridge Regression Confidence Machine , 2001, International Conference on Machine Learning.

[26]  Harris Papadopoulos,et al.  A Soft Computing Approach for Osteoporosis Risk Factor Estimation , 2010, AIAI.

[27]  Harris Papadopoulos,et al.  Assessment of Stroke Risk Based on Morphological Ultrasound Image Analysis with Conformal Prediction , 2010, AIAI.

[28]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[29]  A. H. Murphy.  A New Vector Partition of the Probability Score , 1973 .

[30]  Haris Haralambous,et al.  24-Hour Neural Network Congestion Models for High-Frequency Broadcast Users , 2009, IEEE Transactions on Broadcasting.

[31]  Haris Haralambous,et al.  Reliable prediction intervals with regression neural networks , 2011, Neural Networks.

[32]  Lazaros S. Iliadis,et al.  An Artificial Neural Network model for mountainous water-resources management: The case of Cyprus mountainous watersheds , 2007, Environ. Model. Softw..

[33]  L Edenbrandt,et al.  Intelligent computer reporting 'lack of experience': a confidence measure for decision support systems. , 1998, Clinical physiology.

[34]  John Scott Bridle,et al.  Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition , 1989, NATO Neurocomputing.