V-matrix method of solving statistical inference problems

This paper presents direct settings and rigorous solutions of the main Statistical Inference problems. It shows that rigorous solutions require solving multidimensional Fredholm integral equations of the first kind in the situation where not only the right-hand side of the equation is an approximation, but the operator in the equation is also defined approximately. Using Stefanuyk-Vapnik theory for solving such ill-posed operator equations, constructive methods of empirical inference are introduced. These methods are based on a new concept called V-matrix. This matrix captures geometric properties of the observation data that are ignored by classical statistical methods.

[1]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[2]  Masashi Sugiyama,et al.  Semi-Supervised Learning of Class Balance under Class-Prior Change by Distribution Matching , 2012, ICML.

[3]  Federico Girosi,et al.  Reducing the run-time complexity of Support Vector Machines , 1999 .

[4]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[5]  Masashi Sugiyama,et al.  Change-Point Detection in Time-Series Data by Direct Density-Ratio Estimation , 2009, SDM.

[6]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[7]  Léon Bottou,et al.  Local Algorithms for Pattern Recognition and Dependencies Estimation , 1993, Neural Computation.

[8]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[9]  Takafumi Kanamori,et al.  Density Ratio Estimation in Machine Learning , 2012 .

[10]  Rauf Izmailov,et al.  Learning with Intelligent Teacher: Similarity Control and Knowledge Transfer - In memory of Alexey Chervonenkis , 2015, SLDS.

[11]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection , 2012, J. Mach. Learn. Res..

[12]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[13]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[14]  Alexander Gammerman,et al.  Ridge Regression Learning Algorithm in Dual Variables , 1998, ICML.

[15]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .