Mobile-agent-based distributed variational Bayesian algorithm for density estimation in sensor networks

This study considers the problem of probability density estimation and model order selection in distributed sensor networks. For this purpose, a mobile-agent-based distributed variational Bayesian algorithm is proposed. It is assumed that the measurements can be statistically modelled by a common Gaussian mixture model. In the proposed algorithm, the problems of model order selection and probability density estimation will be considered simultaneously using mobile agents and the variational concept. Initially, considering a component number greater than the true one, the variational Bayesian algorithm will be executed in different nodes. In other words, the mobile agents move through different routes in the network and compute the local sufficient statistics. Afterwards, the global sufficient statistics will be updated using these values and finally the parameters of the probability density function will be calculated. This procedure will be repeated until convergence is reached. At this moment, the component whose mixture probability is lower than a threshold value will be removed. The mentioned steps will continue until the true component number is reached. Convergence of the proposed method will also be analytically studied. Finally, the proposed algorithm will be applied to synthetic and also real-world data sets to show its promising performance.

[1]  D. M. Titterington,et al.  Variational approximations in Bayesian model selection for finite mixture distributions , 2007, Comput. Stat. Data Anal..

[2]  Hairong Qi,et al.  Mobile agent migration modeling and design for target tracking in wireless sensor networks , 2008, Ad Hoc Networks.

[3]  Gonzalo Mateos,et al.  Distributed Recursive Least-Squares: Stability and Performance Analysis , 2011, IEEE Transactions on Signal Processing.

[4]  Behrouz Safarinejadian,et al.  A novel distributed variational approximation method for density estimation in sensor networks , 2016 .

[5]  Kenneth D. Harris,et al.  High-Dimensional Cluster Analysis with the Masked EM Algorithm , 2013, Neural Computation.

[6]  Ali H. Sayed,et al.  Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior , 2013, IEEE Signal Processing Magazine.

[7]  Vladimir Pavlovic,et al.  Decentralized Approximate Bayesian Inference for Distributed Sensor Network , 2016, AAAI.

[8]  Konstantinos Koutroumbas,et al.  A Variational Bayes Framework for Sparse Adaptive Estimation , 2014, IEEE Transactions on Signal Processing.

[9]  Roberto López-Valcarce,et al.  A Diffusion-Based EM Algorithm for Distributed Estimation in Unreliable Sensor Networks , 2013, IEEE Signal Processing Letters.

[10]  Kamil Dedecius,et al.  Diffusion estimation of mixture models with local and global parameters , 2016, 2016 IEEE Statistical Signal Processing Workshop (SSP).

[11]  Stephen J. Roberts,et al.  A tutorial on variational Bayesian inference , 2012, Artificial Intelligence Review.

[12]  Behrooz Safarinejadian,et al.  Distributed Data Clustering Using Mobile Agents and EM Algorithm , 2016, IEEE Systems Journal.

[13]  Dongbing Gu,et al.  Spatial Gaussian Process Regression With Mobile Sensor Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Behrooz Safarinejadian,et al.  Consensus Filter-Based Distributed Variational Bayesian Algorithm for Flow and Speed Density Prediction With Distributed Traffic Sensors , 2017, IEEE Systems Journal.

[15]  Jonathan Walpole,et al.  Storage System Support for Continuous-Media Applications, Part 1: Requirements and Single-Disk Issues , 2004, IEEE Distributed Syst. Online.

[16]  Ali H. Sayed,et al.  Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks , 2011, IEEE Transactions on Signal Processing.

[17]  Hagai Attias,et al.  Inferring Parameters and Structure of Latent Variable Models by Variational Bayes , 1999, UAI.

[18]  John Thompson,et al.  Diffusion-based EM gradient algorithm for density estimation in sensor networks , 2016, 2016 IEEE 17th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[19]  Hairong Qi,et al.  Mobile-agent-based collaborative sensor fusion , 2008, Inf. Fusion.

[20]  Sophie Fosson,et al.  A Distributed Classification/Estimation Algorithm for Sensor Networks , 2014, SIAM J. Control. Optim..

[21]  Dongbing Gu,et al.  Distributed EM Algorithm for Gaussian Mixtures in Sensor Networks , 2008, IEEE Transactions on Neural Networks.

[22]  Chunguang Li,et al.  Distributed Sparse Recursive Least-Squares Over Networks , 2014, IEEE Transactions on Signal Processing.

[23]  Zahia Guessoum.  Adaptive agents and multiagent systems , 2004, IEEE Distributed Systems Online.

[24]  Georgios B. Giannakis,et al.  Distributed Clustering Using Wireless Sensor Networks , 2011, IEEE Journal of Selected Topics in Signal Processing.

[25]  Ali Mohammad-Djafari.  Variational Bayesian Approximation Method for Classification and Clustering with a Mixture of Student-t Model , 2015, GSI.

[26]  Gonzalo Mateos,et al.  Distributed Sparse Linear Regression , 2010, IEEE Transactions on Signal Processing.

[27]  Paolo Braca,et al.  Environmental field estimation by consensus based dynamic sensor networks and underwater gliders , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[28]  Thomas Rodet,et al.  A Measure-Theoretic Variational Bayesian Algorithm for Large Dimensional Problems , 2014, SIAM J. Imaging Sci..

[29]  D.G. Tzikas,et al.  The variational approximation for Bayesian inference , 2008, IEEE Signal Processing Magazine.

[30]  Aristidis Likas,et al.  Unsupervised Learning of Gaussian Mixtures Based on Variational Component Splitting , 2007, IEEE Transactions on Neural Networks.

[31]  Lihua Xie,et al.  Averaging based distributed estimation algorithm for sensor networks with quantized and directed communication , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  Cem Ersoy,et al.  Distributed Mobile Sink Routing for Wireless Sensor Networks: A Survey , 2014, IEEE Communications Surveys & Tutorials.

[33]  Roberto López-Valcarce,et al.  A diffusion-based distributed em algorithm for density estimation in wireless sensor networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[34]  Behrooz Safarinejadian,et al.  A distributed EM algorithm to estimate the parameters of a finite mixture of components , 2009, Knowledge and Information Systems.

[35]  Robert D. Nowak,et al.  Distributed EM algorithms for density estimation and clustering in sensor networks , 2003, IEEE Trans. Signal Process..

[36]  Ian F. Akyildiz,et al.  Sensor Networks , 2002, Encyclopedia of GIS.

[37]  Miguel Lázaro-Gredilla,et al.  Doubly Stochastic Variational Bayes for non-Conjugate Inference , 2014, ICML.

[38]  Henk Wymeersch,et al.  Variational Inference-Based Positioning with Nondeterministic Measurement Accuracies and Reference Location Errors , 2017, IEEE Transactions on Mobile Computing.

[39]  Bernd Sturmfels,et al.  FIXED POINTS OF THE EM ALGORITHM AND NONNEGATIVE RANK BOUNDARIES , 2013, 1312.5634.

[40]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[41]  Behrooz Safarinejadian,et al.  Distributed variational Bayesian algorithms for Gaussian mixtures in sensor networks , 2010, Signal Process..

[42]  D. Titterington,et al.  Convergence properties of a general algorithm for calculating variational Bayesian estimates for a normal mixture model , 2006 .

[43]  Adrian Corduneanu,et al.  Variational Bayesian Model Selection for Mixture Distributions , 2001 .

[44]  Chunguang Li,et al.  Distributed Variational Bayesian Algorithms Over Sensor Networks , 2016, IEEE Transactions on Signal Processing.

[45]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .