Inductive Confidence Machines for Regression

The existing methods of predicting with confidence give good accuracy and confidence values, but quite often are computationally inefficient. Some partial solutions have been suggested in the past. Both the original method and these solutions were based on transductive inference. In this paper we make a radical step of replacing transductive inference with inductive inference and define what we call the Inductive Confidence Machine (ICM); our main concern in this paper is the use of ICM in regression problems. The algorithm proposed in this paper is based on the Ridge Regression procedure (which is usually used for outputting bare predictions) and is much faster than the existing transductive techniques. The inductive approach described in this paper may be the only option available when dealing with large data sets.

[1]  Ming Li,et al.  An Introduction to Kolmogorov Complexity and Its Applications , 1997, Texts in Computer Science.

[2]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[3]  Ilia Nouretdinov,et al.  Algorithmic theory of randomness and its applications , 2003 .

[4]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[5]  Alexander Gammerman,et al.  Machine-Learning Applications of Algorithmic Randomness , 1999, ICML.

[6]  Alexander Gammerman,et al.  Computationally Efficient Transductive Machines , 2000, ALT.

[7]  Vladimir Vovk,et al.  Comparing the Bayes and Typicalness Frameworks , 2001, ECML.

[8]  Alexander Gammerman,et al.  Transductive Confidence Machines for Pattern Recognition , 2002, ECML.

[9]  Alexander Gammerman,et al.  Prediction algorithms and confidence measures based on algorithmic randomness theory , 2002, Theor. Comput. Sci..

[10]  Ilia Nouretdinov,et al.  Transductive Confidence Machine Is Universal , 2003, ALT.

[11]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[12]  D. A. S. Fraser,et al.  Non-Parametric Methods in Statistics. , 1958 .

[13]  Alexander Gammerman,et al.  Transduction with Confidence and Credibility , 1999, IJCAI.

[14]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[15]  Vladimir Vovk,et al.  Ridge Regression Confidence Machine , 2001, International Conference on Machine Learning.