Mean Field Theory for Density Estimation Using Support Vector Machines

This paper presents a novel algorithm for density estimation. This algorithm is based on the support vector machines (SVM) approach and the Mean Field (MF) theory. The SVM decomposes the parameters of the density estimation problem into a quadratic optimization form. This form is suitable for optimization using Mean Field theory. The new algorithm selects the weights of the mixture of kernels used in the SVM estimate more accurately and faster than traditional quadratic programming algorithms. The performance of the proposed algorithms is illustrated using a number of simulated densities. The evaluation shows that the method provides satisfactory results while keeping a reasonable convergence speed.

[1]  Aly A. Farag,et al.  A new unsupervised approach for the classification of multispectral data , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[2]  J. Lamperti.  Probability: A Survey of the Mathematical Theory , 1969 .

[3]  Junbin Gao,et al.  Mean field method for the support vector machine regression , 2003, Neurocomputing.

[4]  A. El-Baz,et al.  Pararmeter estimation in gibbs-markov image models , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[5]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[6]  Aly A. Farag,et al.  Two Sequential Stages Classifier for Multispectral Data , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[7]  Ole Winther,et al.  Gaussian Processes for Classification: Mean-Field Algorithms , 2000, Neural Computation.

[8]  B. Schölkopf,et al.  Advances in kernel methods: support vector learning , 1999 .