Improved fast gauss transform and efficient kernel density estimation

Evaluating sums of multivariate Gaussians is a common computational task in computer vision and pattern recognition, including in the general and powerful kernel density estimation technique. The quadratic computational complexity of the summation is a significant barrier to the scalability of this algorithm to practical applications. The fast Gauss transform (FGT) has successfully accelerated the kernel density estimation to linear running time for low-dimensional problems. Unfortunately, the cost of a direct extension of the FGT to higher-dimensional problems grows exponentially with dimension, making it impractical for dimensions above 3. We develop an improved fast Gauss transform to efficiently estimate sums of Gaussians in higher dimensions, where a new multivariate expansion scheme and an adaptive space subdivision technique dramatically improve the performance. The improved FGT has been applied to the mean shift algorithm achieving linear computational complexity. Experimental results demonstrate the efficiency and effectiveness of our algorithm.

[1]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[2]  David A. Landgrebe,et al.  Fast Parzen Density Estimation Using Clustering-Based Branch and Bound , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  David B. Shmoys,et al.  A Best Possible Heuristic for the k-Center Problem , 1985, Math. Oper. Res..

[4]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[5]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[6]  Leslie Greengard,et al.  The Fast Gauss Transform , 1991, SIAM J. Sci. Comput..

[7]  T. Cacoullos.  Estimation of a multivariate density , 1966 .

[8]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[9]  John Strain,et al.  The Fast Gauss Transform with Variable Scales , 1991, SIAM J. Sci. Comput..

[10]  Ramani Duraiswami,et al.  Data Structures, Optimal Choice of Parameters, and Complexity Results for Generalized Multilevel Fast Multipole Methods in $d$ Dimensions , 2003 .

[11]  D. Eppstein,et al.  Approximation algorithms for geometric problems , 1996 .

[12]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  David G. Stork,et al.  Pattern Classification , 1973 .

[14]  Jack-Gérard Postaire,et al.  A Fast Algorithm for Nonparametric Probability Density Estimation , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Teofilo F. GONZALEZ,et al.  Clustering to Minimize the Maximum Intercluster Distance , 1985, Theor. Comput. Sci..

[16]  Peter J. Huber,et al.  Robust Statistical Procedures: Second Edition , 1996 .

[17]  B. Silverman,et al.  Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[18]  B. Silverman,et al.  Algorithm AS 176: Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[19]  Tomás Feder,et al.  Optimal algorithms for approximate clustering , 1988, STOC '88.

[20]  Dorit S. Hochbaum,et al.  Approximation Algorithms for NP-Hard Problems , 1996 .

[21]  L. Greengard,et al.  A fast algorithm for the evaluation of heat potentials , 1990 .

[22]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Luc Devroye,et al.  Data Structures in Kernel Density Estimation , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Larry S. Davis,et al.  Mean-shift analysis using quasiNewton methods , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[25]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[26]  Larry S. Davis,et al.  Efficient non-parametric adaptive color modeling using fast Gauss transform , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Leslie Greengard,et al.  A fast algorithm for particle simulations , 1987 .

[28]  P. J. Huber.  Robust Statistical Procedures , 1977 .

[29]  George Roussos,et al.  A New Error Estimate of the Fast Gauss Transform , 2002, SIAM J. Sci. Comput..

[30]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Keinosuke Fukunaga,et al.  The Reduced Parzen Classifier , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.