A sufficient condition for the convergence of the mean shift algorithm with Gaussian kernel

The mean shift (MS) algorithm is a non-parametric, iterative technique that has been used to find modes of an estimated probability density function (pdf). Although the MS algorithm has been widely used in many applications, such as clustering, image segmentation, and object tracking, a rigorous proof for its convergence is still missing. This paper tries to fill some of the gaps between theory and practice by presenting specific theoretical results about the convergence of the MS algorithm. To achieve this goal, first we show that all the stationary points of an estimated pdf using a certain class of kernel functions are inside the convex hull of the data set. Then the convergence of the sequence generated by the MS algorithm for an estimated pdf with isolated stationary points will be proved. Finally, we present a sufficient condition for the estimated pdf using the Gaussian kernel to have isolated stationary points.

[1]  Haifeng Chen,et al.  Robust Fusion of Uncertain Information , 2003, 2003 Conference on Computer Vision and Pattern Recognition Workshop.

[2]  Kanti B. Datta,et al.  Matrix and linear algebra , 1991 .

[3]  Qingshan Liu,et al.  Mean Shift-Based Edge Detection for Color Image , 2005, 2005 International Conference on Neural Networks and Brain.

[4]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[5]  Youness Aliyari Ghassabeh,et al.  On the convergence of the mean shift algorithm in the one-dimensional space , 2013, Pattern Recognit. Lett..

[6]  Tamás Linder,et al.  On the convergence and applications of mean shift type algorithms , 2012, 2012 25th IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[7]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[8]  Adam Ostaszewski,et al.  Advanced Mathematical Methods , 1990 .

[9]  G. Takahara,et al.  On noisy source vector quantization via a subspace constrained mean shift algorithm , 2012, 2012 26th Biennial Symposium on Communications (QBSC).

[10]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Bo Thiesson,et al.  Image and Video Segmentation by Anisotropic Kernel Mean Shift , 2004, ECCV.

[12]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[13]  Gerald Schaefer,et al.  Bayesian image segmentation with mean shift , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[14]  Surajit Ray,et al.  The topography of multivariate normal mixtures , 2005 .

[15]  Shai Avidan.  Ensemble Tracking , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Miguel Á. Carreira-Perpiñán,et al.  Gaussian Mean-Shift Is an EM Algorithm , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Zhanyi Hu,et al.  A note on the convergence of the mean shift , 2007, Pattern Recognit..

[18]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[19]  Gerhard Tutz,et al.  The fitting of multifunctions : an approach to nonparametric multimodal regression. , 2006 .

[20]  Benjamin Wallace.  On the Critical Points of Gaussian Mixtures , 2013 .

[21]  Miguel Á. Carreira-Perpiñán,et al.  On the Number of Modes of a Gaussian Mixture , 2003, Scale-Space.

[22]  G. Styan,et al.  Equalities and Inequalities for Ranks of Matrices , 1974 .

[23]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Tamás Linder,et al.  On some convergence properties of the subspace constrained mean shift , 2013, Pattern Recognit..

[25]  Surajit Ray,et al.  A Nonparametric Statistical Approach to Clustering via Mode Identification , 2007, J. Mach. Learn. Res..

[26]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[27]  R. A. Boyles.  On the Convergence of the EM Algorithm , 1983 .

[28]  Youness Aliyari Ghassabeh.  Asymptotic stability of equilibrium points of mean shift algorithm , 2014, Machine Learning.

[29]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[30]  Naixue Xiong,et al.  Edge Detection Based on Fast Adaptive Mean Shift Algorithm , 2009, 2009 International Conference on Computational Science and Engineering.

[31]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Carlo Tomasi,et al.  Mean shift is a bound optimization , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.