Kernel Smoothing, Mean Shift, and Their Learning Theory with Directional Data

Directional data consist of observations distributed on a (hyper)sphere, and appear in many applied fields, such as astronomy, ecology, and environmental science. This paper studies both statistical and computational problems of kernel smoothing for directional data. We generalize the classical mean shift algorithm to directional data, which allows us to identify local modes of the directional kernel density estimator (KDE). The statistical convergence rates of the directional KDE and its derivatives are derived, and the problem of mode estimation is examined. We also prove the ascending property of our directional mean shift algorithm and investigate a general problem of gradient ascent on the unit hypersphere. To demonstrate the applicability of our proposed algorithm, we evaluate it as a mode clustering method on both simulated and real-world datasets.

[1]  R. Nichol,et al.  The Dark Energy Survey: more than dark energy - an overview , 2016, 1601.00329.

[2]  Larry A. Wasserman,et al.  Nonparametric Ridge Estimation , 2012, ArXiv.

[3]  L. Wasserman,et al.  Enhanced Mode Clustering , 2014 .

[4]  Miguel Á. Carreira-Perpiñán,et al.  Gaussian Mean-Shift Is an EM Algorithm , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Alberto Rodríguez-Casal,et al.  A plug-in rule for bandwidth selection in circular density estimation , 2012, Comput. Stat. Data Anal..

[6]  Silvere Bonnabel,et al.  Stochastic Gradient Descent on Riemannian Manifolds , 2011, IEEE Transactions on Automatic Control.

[7]  Uwe Einmahl,et al.  Uniform in bandwidth consistency of kernel-type function estimators , 2005 .

[8]  M. Morse Relations between the critical points of a real function of $n$ independent variables , 1925 .

[9]  Suvrit Sra,et al.  First-order Methods for Geodesically Convex Optimization , 2016, COLT.

[10]  Charles C. Taylor,et al.  Automatic bandwidth selection for circular density estimation , 2008, Comput. Stat. Data Anal..

[11]  John P. Snyder,et al.  An Album of Map Projections , 1989 .

[12]  Yen-Chi Chen,et al.  A tutorial on kernel density estimation and recent advances , 2017, 1704.03924.

[13]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[14]  E. Parzen On Estimation of a Probability Density Function and Mode , 1962 .

[15]  M. Skrutskie,et al.  The Two Micron All Sky Survey (2MASS) , 2006 .

[16]  John M. Lee Introduction to Smooth Manifolds , 2002 .

[17]  Zhanyi Hu,et al.  A note on the convergence of the mean shift , 2007, Pattern Recognit..

[18]  Inderjit S. Dhillon,et al.  Clustering on the Unit Hypersphere using von Mises-Fisher Distributions , 2005, J. Mach. Learn. Res..

[19]  V. Rovenski,et al.  Differential Geometry of Curves and Surfaces , 1952, Nature.

[20]  Larry Wasserman,et al.  All of Nonparametric Statistics (Springer Texts in Statistics) , 2006 .

[21]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[23]  W. Rudin Principles of mathematical analysis , 1964 .

[24]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[25]  R. Smullyan ANNALS OF MATHEMATICS STUDIES , 1961 .

[26]  Z. Chengqing,et al.  Central limit theorem for integrated square error of kernel estimators of spherical density , 2001 .

[27]  M. Taylor,et al.  Active structures of the Himalayan-Tibetan orogen and their relationships to earthquake distribution, contemporary strain field, and Cenozoic volcanism , 2009 .

[28]  Gabriela Hug,et al.  Projected gradient descent on Riemannian manifolds with applications to online power system optimization , 2016, 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[29]  E. Giné,et al.  Rates of strong uniform consistency for multivariate kernel density estimators , 2002 .

[30]  C. Frye,et al.  Spherical Harmonics in p Dimensions , 2012, 1205.3548.

[31]  O. Hyrien,et al.  Fast Nonparametric Density-Based Clustering of Large Datasets Using a Stochastic Approximation Mean-Shift Algorithm , 2016, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[32]  Ming Tang,et al.  Accelerated Convergence Using Dynamic Mean Shift , 2006, ECCV.

[33]  Jussi Klemelä,et al.  Estimation of Densities and Derivatives of Densities with Directional Data , 2000 .

[34]  Joseph P. Romano On weak convergence and optimality of kernel density estimates of the mode , 1988 .

[35]  Wenceslao González-Manteiga,et al.  Kernel density estimation for directional-linear data , 2012, J. Multivar. Anal..

[36]  M. Wand,et al.  ASYMPTOTICS FOR GENERAL MULTIVARIATE KERNEL DENSITY DERIVATIVE ESTIMATORS , 2011 .

[37]  Yu. D. Burago,et al.  A.D. Alexandrov spaces with curvature bounded below , 1992 .

[38]  M. Talagrand New concentration inequalities in product spaces , 1996 .

[39]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[40]  E. al.,et al.  The Sloan Digital Sky Survey: Technical summary , 2000, astro-ph/0006396.

[41]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[43]  Eduardo Garc'ia-Portugu'es,et al.  Exact risk improvement of bandwidth selectors for kernel density estimation with directional data , 2013, 1306.0517.

[44]  Kellen Petersen August Real Analysis , 2009 .

[45]  Thomas Verdebout,et al.  Applied Directional Statistics , 2018 .

[46]  Youness Aliyari Ghassabeh,et al.  A sufficient condition for the convergence of the mean shift algorithm with Gaussian kernel , 2015, J. Multivar. Anal..

[47]  Marston Morse The Foundations of a Theory of the Calculus of Variations in the Large in m-Space (Second Paper) , 1930 .

[48]  L. Soderblom,et al.  Martian planetwide crater distributions - Implications for geologic history and surface processes , 1974 .

[49]  Stan Z. Li,et al.  Stochastic gradient kernel density mode-seeking , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  M. Rosenblatt Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[51]  Suvrit Sra,et al.  A short note on parameter approximation for von Mises-Fisher distributions: and a fast implementation of Is(x) , 2012, Comput. Stat..

[52]  Miguel Á. Carreira-Perpiñán,et al.  Generalised blurring mean-shift algorithms for nonparametric clustering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  D. Pollard,et al.  $U$-Processes: Rates of Convergence , 1987 .

[54]  Nadine G. Barlow,et al.  Constraining geologic properties and processes through the use of impact craters , 2015 .

[55]  David Mason,et al.  On the Estimation of the Gradient Lines of a Density and the Consistency of the Mean-Shift Algorithm , 2016, J. Mach. Learn. Res..

[56]  Sébastien Bubeck,et al.  Convex Optimization: Algorithms and Complexity , 2014, Found. Trends Mach. Learn..

[57]  Youness Aliyari Ghassabeh,et al.  On the convergence of the mean shift algorithm in the one-dimensional space , 2013, Pattern Recognit. Lett..

[58]  Philippe Vieu,et al.  A note on density mode estimation , 1996 .

[59]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[60]  P. Hall,et al.  Kernel density estimation with spherical data , 1987 .

[61]  Charles C. Taylor,et al.  Kernel density estimation on the torus , 2011 .

[62]  Miguel Á. Carreira-Perpiñán,et al.  A review of mean-shift algorithms for clustering , 2015, ArXiv.

[63]  K. Priestley,et al.  Earthquake distribution patterns in Africa: their relationship to variations in lithospheric and geological structure, and their rheological implications , 2011 .

[64]  Martin J. Wainwright,et al.  Statistical guarantees for the EM algorithm: From population to sample-based analysis , 2014, ArXiv.

[65]  Calyampudi R. Rao,et al.  Kernel estimators of density function of directional data , 1988 .

[66]  Miguel Á. Carreira-Perpiñán,et al.  Fast nonparametric clustering with Gaussian blurring mean-shift , 2006, ICML.

[68]  Robert E. Mahony,et al.  An Extrinsic Look at the Riemannian Hessian , 2013, GSI.