Distributed Splitting-Over-Features Sparse Bayesian Learning with Alternating Direction Method of Multipliers

In processing spatially distributed data, multi-agent robotic platforms equipped with sensors and computing capabilities are gaining interest for applications in inhospitable environments. In this work an algorithm for a distributed realization of sparse bayesian learning (SBL) is discussed for learning a static spatial process with the splitting-over-features approach over a network of interconnected agents. The observed process is modeled as a superposition of weighted kernel functions, or features as we call it, centered at the agent's measurement locations. SBL is then used to determine which feature is relevant for representing the spatial process. Using upper bounding convex functions, the SBL parameter estimation is formulated as $\ell_{1}$-norm constrained optimization, which is solved distributively using alternating direction method of multipliers (ADMM) and averaged consensus. The performance of the method is demonstrated by processing real magnetic field data collected in a laboratory.

[1]  Bhaskar D. Rao,et al.  Type I and Type II Bayesian Methods for Sparse Signal Recovery Using Scale Mixtures , 2015, IEEE Transactions on Signal Processing.

[2]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[3]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[4]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[5]  Bhaskar D. Rao,et al.  Latent Variable Bayesian Models for Promoting Sparsity , 2011, IEEE Transactions on Information Theory.

[6]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[7]  John Mark,et al.  Introduction to radial basis function networks , 1996 .

[8]  H. Vincent Poor,et al.  Attribute-Distributed Learning: Models, Limits, and Algorithms , 2011, IEEE Transactions on Signal Processing.

[9]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[10]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[12]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[13]  Richard M. Murray,et al.  Consensus problems in networks of agents with switching topology and time-delays , 2004, IEEE Transactions on Automatic Control.

[14]  H. Vincent Poor,et al.  Dimensionally distributed learning models and algorithm , 2008, 2008 11th International Conference on Information Fusion.

[15]  David P. Wipf,et al.  A New View of Automatic Relevance Determination , 2007, NIPS.

[16]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[17]  G. Sohie,et al.  Generalization of the matrix inversion lemma , 1986, Proceedings of the IEEE.

[18]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[19]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .