Robust Fast Online Multivariate Non-parametric Density Estimator

With the recent development of network and sensor technologies, vast amounts of data are being continuously generated in real time from real-world environments. Such data includes in many noise, and it is not easy to predict that distribution underlying the data in advance. Probability density estimation is a critical task of machine learning, but it is difficult to accomplish it for big data in the real world. For handling such data, we propose a robust fast online multivariate non-parametric density estimator. Our proposed method extends the kernel density estimation and Self-Organizing Incremental Neural Network. The experimental results show that our proposed method outperforms or achieves a state-of-the-art performance.

[1]  Shen Furao,et al.  A general associative memory based on self-organizing incremental neural network , 2013, Neurocomputing.

[2]  Juan Miguel Ortiz-de-Lazcano-Lobato,et al.  Soft clustering for nonparametric probability density function estimation , 2007, Pattern Recognit. Lett..

[3]  Winston A Hide,et al.  Big data: The future of biocuration , 2008, Nature.

[4]  Shen Furao,et al.  An enhanced self-organizing incremental neural network for online unsupervised learning , 2007, Neural Networks.

[5]  Marcel J. T. Reinders,et al.  Integration of prior knowledge of measurement noise in kernel density classification , 2008, Pattern Recognit..

[6]  Aram Kawewong,et al.  Online incremental attribute-based zero-shot learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  M. C. Jones,et al.  On optimal data-based bandwidth selection in kernel density estimation , 1991 .

[8]  Osamu Hasegawa,et al.  Associative Memory for Online Learning in Noisy Environments Using Self-Organizing Incremental Neural Network , 2009, IEEE Transactions on Neural Networks.

[9]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[10]  Mineichi Kudo,et al.  Data compression by volume prototypes for streaming data , 2010, Pattern Recognit..

[11]  Tat-Jun Chin,et al.  Incremental Kernel Principal Component Analysis , 2007, IEEE Transactions on Image Processing.

[12]  Wei Fan,et al.  Mining big data: current status, and forecast to the future , 2013, SKDD.

[13]  Chao He,et al.  Probability Density Estimation from Optimally Condensed Data Samples , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Toyoaki Nishida,et al.  Online incremental clustering with distance metric learning for high dimensional data , 2011, The 2011 International Joint Conference on Neural Networks.

[15]  Shen Furao,et al.  A fast nearest neighbor classifier based on self-organizing incremental neural network , 2008, Neural Networks.

[16]  Tong Zhang,et al.  Improved Local Coordinate Coding using Local Tangents , 2010, ICML.

[17]  Shigeo Abe,et al.  An Efficient Incremental Kernel Principal Component Analysis for Online Feature Selection , 2007, 2007 International Joint Conference on Neural Networks.

[18]  M. C. Jones,et al.  A Brief Survey of Bandwidth Selection for Density Estimation , 1996 .

[19]  Katharine Armstrong,et al.  Big data: a revolution that will transform how we live, work, and think , 2014 .

[20]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[21]  Yasue Mitsukura,et al.  Fast Incremental Algorithm of Simple Principal Component Analysis (特集 若手研究者) -- (ソフトコンピューティング・学習) , 2009 .

[22]  C. A. Murthy,et al.  A data driven procedure for density estimation with some applications , 1996, Pattern Recognit..

[23]  Daniela Rodriguez,et al.  Kernel Density Estimation on Riemannian Manifolds: Asymptotic Results , 2009, Journal of Mathematical Imaging and Vision.

[24]  Shen Furao,et al.  Online Knowledge Acquisition and General Problem Solving in a Real World by Humanoid Robots , 2010, ICANN.

[25]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[26]  Clayton D. Scott,et al.  Robust kernel density estimation , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[28]  Danijel Skocaj,et al.  Multivariate online kernel density estimation with Gaussian kernels , 2011, Pattern Recognit..

[29]  Justus H. Piater,et al.  Online Learning of Gaussian Mixture Models - a Two-Level Approach , 2008, VISAPP.

[30]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[31]  Anne Lohrli.  Chapman and Hall , 1985 .

[32]  Imad Aad,et al.  The Mobile Data Challenge: Big Data for Mobile Computing Research , 2012 .

[33]  Aram Kawewong,et al.  Incremental Learning Framework for Indoor Scene Recognition , 2013, AAAI.

[34]  A. Bowman.  An alternative method of cross-validation for the smoothing of density estimates , 1984 .

[35]  Bruno Pelletier.  Kernel density estimation on Riemannian manifolds , 2005 .

[36]  Zhaohong Deng,et al.  FRSDE: Fast reduced set density estimator using minimal enclosing ball approximation , 2008, Pattern Recognit..

[37]  Shen Furao,et al.  An incremental network for on-line unsupervised classification and topology learning , 2006, Neural Networks.

[38]  Mia Hubert,et al.  ROBPCA: A New Approach to Robust Principal Component Analysis , 2005, Technometrics.