Nonparametric multivariate density estimation: a comparative study

The paper algorithmically and empirically studies two major types of nonparametric multivariate density estimation techniques, where no assumption is made about the data being drawn from any of known parametric families of distribution. The first type is the popular kernel method (and several of its variants) which uses locally tuned radial basis (e.g., Gaussian) functions to interpolate the multidimensional density; the second type is based on an exploratory projection pursuit technique which interprets the multidimensional density through the construction of several 1D densities along highly "interesting" projections of multidimensional data. Performance evaluations using training data from mixture Gaussian and mixture Cauchy densities are presented. The results show that the curse of dimensionality and the sensitivity of control parameters have a much more adverse impact on the kernel density estimators than on the projection pursuit density estimators. >

[1]  P. Hall.  On Polynomial-Based Projection Indices for Exploratory Projection Pursuit , 1989 .

[2]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[3]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[4]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[5]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[6]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[7]  Andreas Buja,et al.  Analyzing High-Dimensional Data with Motion Graphics , 1990, SIAM J. Sci. Comput..

[8]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[9]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[10]  Robert M. Gray,et al.  Combining image classification and image compression using vector quantization , 1993, [Proceedings] DCC `93: Data Compression Conference.

[11]  Ian Abramson.  On Bandwidth Variation in Kernel Estimates-A Square Root Law , 1982 .

[12]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[13]  Kris Popat,et al.  Cluster-based probability model applied to image restoration and compression , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[15]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[16]  J. Friedman,et al.  PROJECTION PURSUIT DENSITY ESTIMATION , 1984 .

[17]  R. Gray,et al.  Using vector quantization for image processing , 1993, Proc. IEEE.

[18]  Jenq-Neng Hwang,et al.  Robust construction of radial basis function networks for classification , 1993, IEEE International Conference on Neural Networks.

[19]  Jenq-Neng Hwang,et al.  Unsupervised learning for multivariate probability density estimation: radial basis and projection pursuit , 1993, IEEE International Conference on Neural Networks.

[20]  Geoffrey J. McLachlan,et al.  Mixture models : inference and applications to clustering , 1989 .

[21]  Rabab Kreidieh Ward,et al.  Vector Quantization Technique for Nonparametric Classifier Design , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[23]  Kris Popat,et al.  Novel cluster-based probability model for texture synthesis, classification, and compression , 1993, Other Conferences.

[24]  Peter E. Hart,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[25]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[26]  S. Klinke,et al.  Exploratory Projection Pursuit , 1995 .