A Branch and Bound Algorithm for Finding the Modes in Kernel Density Estimates

Kernel density estimators are established tools in nonparametric statistics. Due to their flexibility and ease of use, these methods are popular in computer vision and pattern recognition for tasks such as object tracking in video or image segmentation. The most frequently used algorithm for finding the modes in such densities (the mean shift) is a gradient ascent rule, which converges to local maxima. We propose a novel, globally optimal branch and bound algorithm for finding the modes in kernel densities. We show in experiments on datasets up to dimension five that the branch and bound method is faster than local optimization and observe linear runtime scaling of our method with sample size. Quantitative experiments on simulated data show that our method gives statistically significantly more accurate solutions than the mean shift. The mode localization accuracy is about five times more precise than that of the mean shift for all tested parameters. Applications to color image segmentation on an established benchmark test set also show measurably improved results when using global optimization.

[1]  Keinosuke Fukunaga,et al.  A Branch and Bound Clustering Algorithm , 1975, IEEE Transactions on Computers.

[2]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[3]  Miguel Á. Carreira-Perpiñán,et al.  Gaussian Mean-Shift Is an EM Algorithm , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  F. Frances Yao,et al.  Computational Geometry , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[5]  Anton van den Hengel,et al.  Fast global kernel density mode seeking with application to localization and tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Thomas M. Breuel.  On the use of interval arithmetic in geometric branch and bound algorithms , 2003, Pattern Recognit. Lett..

[7]  Christoph H. Lampert,et al.  Satellite Tracks Removal in Astronomical Images , 2006, CIARP.

[8]  Thomas M. Breuel,et al.  Global Modes in Kernel Density Estimation: RAST Clustering , 2007, 7th International Conference on Hybrid Intelligent Systems (HIS 2007).

[9]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[10]  David A. Landgrebe,et al.  Fast Parzen Density Estimation Using Clustering-Based Branch and Bound , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Robert T. Collins,et al.  Mean-shift blob tracking through scale space , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  Francisco Sandoval Hernández,et al.  Mean shift based clustering of Hough domain for fast line segment detection , 2006, Pattern Recognit. Lett..

[13]  Dorin Comaniciu,et al.  Mean shift and optimal prediction for efficient object tracking , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[14]  Eldon Hansen,et al.  Global optimization using interval analysis , 1992, Pure and applied mathematics.

[15]  Thomas M. Breuel,et al.  Finding lines under bounded error , 1996, Pattern Recognit..

[16]  Thomas M. Breuel,et al.  Fast recognition using adaptive subdivisions of transformation space , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Larry S. Davis,et al.  Improved fast gauss transform and efficient kernel density estimation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Ilan Shimshoni,et al.  Mean shift based clustering in high dimensions: a texture classification example , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[21]  Christoph H. Lampert,et al.  Optimal Dominant Motion Estimation Using Adaptive Search of Transformation Space , 2007, DAGM-Symposium.

[22]  Jack-Gérard Postaire,et al.  A Fast Algorithm for Nonparametric Probability Density Estimation , 1982, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Visvanathan Ramesh,et al.  Prior-Constrained Scale-Space Mean Shift , 2006, BMVC.

[24]  Dorin Comaniciu.  Image segmentation using clustering with saddle point detection , 2002, Proceedings. International Conference on Image Processing.

[25]  Dorin Comaniciu,et al.  The Variable Bandwidth Mean Shift and Data-Driven Scale Selection , 2001, ICCV.

[26]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[27]  T. Breuel,et al.  Electronic Letters on Computer Vision and Image Analysis 6(1):44-54, 2007 Optimal Geometric Matching for Patch-Based Object Detection , 2006 .

[28]  Dorin Comaniciu,et al.  Mean shift analysis and applications , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.