Distributed variational sparse Bayesian learning for sensor networks

In this work we present a distributed sparse Bayesian learning (dSBL) regression algorithm. It can be used for collaborative sparse estimation of spatial functions in wireless sensor networks (WSNs). The sensor measurements are modeled as a weighted superposition of basis functions. When kernels are used, the algorithm forms a distributed version of the relevance vector machine. The proposed method is based on a combination of variational inference and loopy belief propagation, where data is only communicated between neighboring nodes without the need for a fusion center. We show that for tree structured networks, under certain parameterization, dSBL coincides with centralized sparse Bayesian learning (cSBL). For general loopy networks, dSBL and cSBL are differend, yet simulations show much faster convergence over the variational inference iterations at similar sparsity and mean squared error performance. Furthermore, compared to other sparse distributed regression methods, our method does not require any cross-tuning of sparsity parameters.

[1]  Benjamin Van Roy,et al.  Consensus Propagation , 2005, IEEE Transactions on Information Theory.

[2]  Stephen P. Boyd,et al.  A scheme for robust distributed sensor fusion based on average consensus , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[3]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[4]  Christopher M. Bishop,et al.  Variational Relevance Vector Machines , 2000, UAI.

[5]  Bhaskar D. Rao,et al.  Perspectives on Sparse Bayesian Learning , 2003, NIPS.

[6]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[7]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[8]  Gonzalo Mateos,et al.  Distributed Sparse Linear Regression , 2010, IEEE Transactions on Signal Processing.

[9]  Dongbing Gu,et al.  Distributed regression over sensor networks: An support vector machine approach , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[11]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[12]  Ananthram Swami,et al.  Wireless Sensor Networks: Signal Processing and Communications , 2007 .

[13]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[14]  Dmitry M. Malioutov,et al.  Walk-Sums and Belief Propagation in Gaussian Graphical Models , 2006, J. Mach. Learn. Res..

[15]  Hichem Snoussi,et al.  Distributed Regression in Sensor Networks with a Reduced-Order Kernel Model , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[16]  Dmitriy Shutin,et al.  Sparse Bayesian consensus-based distributed field estimation , 2011, 2011 5th International Conference on Signal Processing and Communication Systems (ICSPCS).

[17]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[18]  H. Vincent Poor,et al.  Distributed learning in wireless sensor networks , 2005, IEEE Signal Processing Magazine.

[19]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[20]  William T. Freeman,et al.  Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology , 1999, Neural Computation.

[21]  Petar M. Djuric,et al.  Distributed Bayesian learning in multiagent systems: Improving our understanding of its capabilities and limitations , 2012, IEEE Signal Processing Magazine.

[22]  Vasilis Friderikos,et al.  Cross-Layer Optimization to Maximize Fairness Among TCP Flows of Different TCP Flavors , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[23]  Qing Zhao,et al.  Distributed Learning in Wireless Sensor Networks , 2007 .