Clustering using sum-of-norms regularization: With application to particle filter output computation

We present a novel clustering method, formulated as a convex optimization problem. The method is based on over-parameterization and uses a sum-of-norms (SON) regularization to control the tradeoff between the model fit and the number of clusters. Hence, the number of clusters can be automatically adapted to best describe the data, and need not to be specified a priori. We apply SON clustering to cluster the particles in a particle filter, an application where the number of clusters is often unknown and time varying, making SON clustering an attractive alternative.

[1]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[2]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[3]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[4]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[5]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[6]  Pedro Larrañaga,et al.  An empirical comparison of four initialization methods for the K-Means algorithm , 1999, Pattern Recognit. Lett..

[7]  Javier Nicolás Sánchez,et al.  Robust global localization using clustered particle filtering , 2002, AAAI/IAAI.

[8]  Petar M. Djuric,et al.  Gaussian sum particle filtering , 2003, IEEE Trans. Signal Process..

[9]  Patrick Pérez,et al.  Maintaining multimodality through mixture tracking , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[11]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[12]  Parameswaran Ramanathan,et al.  Distributed particle filter with GMM approximation for multiple targets localization and tracking in wireless sensor network , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[13]  Kunio Takezawa Whiley Series in Probability and Statistics , 2005 .

[14]  J. Suykens,et al.  Convex Clustering Shrinkage , 2005 .

[15]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[16]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[17]  Thomas B. Schön,et al.  Fast particle filters for multi-rate sensors , 2007, 2007 15th European Signal Processing Conference.

[18]  Simon J. Godsill,et al.  An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo , 2007, Proceedings of the IEEE.

[19]  Polina Golland,et al.  Convex Clustering with Exemplar-Based Models , 2007, NIPS.

[20]  Sebastian Nowozin,et al.  A decoupled approach to exemplar-based unsupervised learning , 2008, ICML '08.

[21]  Jacob Roll,et al.  Piecewise linear solution paths with application to direct weight optimization , 2008, Autom..

[22]  G. McLachlan,et al.  The EM Algorithm and Extensions: Second Edition , 2008 .

[23]  Stephen P. Boyd,et al.  Graph Implementations for Nonsmooth Convex Programs , 2008, Recent Advances in Learning and Control.

[24]  Stephen P. Boyd,et al.  1 Trend Filtering , 2009, SIAM Rev..

[25]  Fredrik Gustafsson,et al.  Road target tracking with an approximative Rao-Blackwellized Particle Filter , 2009, 2009 12th International Conference on Information Fusion.

[26]  Stephen P. Boyd,et al.  Segmentation of ARX-models using sum-of-norms regularization , 2010, Autom..

[27]  L. Ljung,et al.  Just Relax and Come Clustering! : A Convexification of k-Means Clustering , 2011 .