A Multitask Diffusion Strategy With Optimized Inter-cluster Cooperation

We consider a multitask estimation problem where nodes in a network are divided into several connected clusters, with each cluster performing a least-mean-squares estimation of a different random parameter vector. Inspired by the adapt-then-combine diffusion strategy, we propose a multitask diffusion strategy whose mean stability can be ensured whenever individual nodes are stable in the mean, regardless of the inter-cluster cooperation weights. In addition, the proposed strategy is able to achieve an asymptotically unbiased estimation when the parameters have same mean. We also develop an inter-cluster cooperation weights selection scheme that allows each node in the network to locally optimize its inter-cluster cooperation weights. Numerical results demonstrate that our approach leads to a lower average steady-state network mean-square deviation, compared with using weights selected by various other commonly adopted methods in the literature.

[1]  Wee Peng Tay,et al.  Multi-Hop Diffusion LMS for Energy-Constrained Distributed Estimation , 2015, IEEE Transactions on Signal Processing.

[2]  Ali H. Sayed,et al.  Diffusion Adaptation over Networks , 2012, ArXiv.

[3]  Kostas Berberidis,et al.  Coalitional game theoretic approach to distributed adaptive parameter estimation , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Jie Chen,et al.  Diffusion LMS Over Multitask Networks , 2014, IEEE Transactions on Signal Processing.

[5]  Jie Chen,et al.  Diffusion LMS for clustered multitask networks , 2013, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Jie Chen,et al.  Group diffusion LMS , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Yi Zhang,et al.  Distributed boundary estimation for spectrum sensing in cognitive radio networks , 2013, 2013 IEEE Wireless Communications and Networking Conference (WCNC).

[8]  Stephen P. Boyd,et al.  A space-time diffusion scheme for peer-to-peer least-squares estimation , 2006, 2006 5th International Conference on Information Processing in Sensor Networks.

[9]  Hyundong Shin,et al.  Distributed Local Linear Parameter Estimation Using Gaussian SPAWN , 2014, IEEE Transactions on Signal Processing.

[10]  Kostas Berberidis,et al.  Distributed Incremental-Based LMS for Node-Specific Adaptive Parameter Estimation , 2014, IEEE Transactions on Signal Processing.

[11]  Jae-Woo Lee,et al.  A Variable Step-Size Diffusion LMS Algorithm for Distributed Estimation , 2015, IEEE Transactions on Signal Processing.

[12]  Sergios Theodoridis,et al.  Adaptive Robust Distributed Learning in Diffusion Sensor Networks , 2011, IEEE Transactions on Signal Processing.

[13]  Cédric Richard,et al.  Multitask Diffusion Adaptation Over Asynchronous Networks , 2014, IEEE Transactions on Signal Processing.

[14]  Jie Chen,et al.  Multitask Diffusion Adaptation Over Networks , 2013, IEEE Transactions on Signal Processing.

[15]  Asuman E. Ozdaglar,et al.  Constrained Consensus and Optimization in Multi-Agent Networks , 2008, IEEE Transactions on Automatic Control.

[16]  Ali H. Sayed,et al.  A strategy for adjusting combination weights over adaptive networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Ali H. Sayed,et al.  Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior , 2013, IEEE Signal Processing Magazine.

[18]  Ali H. Sayed,et al.  Diffusion Adaptation Over Networks Under Imperfect Information Exchange and Non-Stationary Data , 2011, IEEE Transactions on Signal Processing.

[19]  Marc Moonen,et al.  Unsupervised diffusion-based LMS for node-specific parameter estimation over wireless sensor networks , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  François Quitin,et al.  Distributed Localization of a RF Target in NLOS Environments , 2015, IEEE Journal on Selected Areas in Communications.

[21]  Ali H. Sayed,et al.  Adjustment of combination weights over adaptive diffusion networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Ali H. Sayed,et al.  Performance Limits for Distributed Estimation Over LMS Adaptive Networks , 2012, IEEE Transactions on Signal Processing.

[23]  Yuan Wang,et al.  An energy-efficient diffusion strategy over adaptive networks , 2015, 2015 10th International Conference on Information, Communications and Signal Processing (ICICS).

[24]  Cédric Richard,et al.  Multitask diffusion LMS with sparsity-based regularization , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  Yuan Wang,et al.  Multitask diffusion LMS with optimized inter-cluster cooperation , 2016, 2016 IEEE Statistical Signal Processing Workshop (SSP).

[26]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[27]  Dimitri P. Bertsekas,et al.  A New Class of Incremental Gradient Methods for Least Squares Problems , 1997, SIAM J. Optim..

[28]  Ali H. Sayed,et al.  Adaptive Networks , 2014, Proceedings of the IEEE.

[29]  Kostas Berberidis,et al.  Distributed Diffusion-Based LMS for Node-Specific Adaptive Parameter Estimation , 2014, IEEE Transactions on Signal Processing.

[30]  Ali H. Sayed,et al.  Diffusion LMS Strategies for Distributed Estimation , 2010, IEEE Transactions on Signal Processing.

[31]  Edwin K. P. Chong,et al.  Robust Decentralized Detection and Social Learning in Tandem Networks , 2015, IEEE Transactions on Signal Processing.

[32]  Gang George Yin,et al.  Distributed Energy-Aware Diffusion Least Mean Squares: Game-Theoretic Learning , 2013, IEEE Journal of Selected Topics in Signal Processing.

[33]  Suleyman Serdar Kozat,et al.  Compressive Diffusion Strategies Over Distributed Networks for Reduced Communication Load , 2014, IEEE Transactions on Signal Processing.

[34]  Ali H. Sayed,et al.  Adaptive Penalty-Based Distributed Stochastic Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[35]  Ali H. Sayed,et al.  Diffusion Strategies Outperform Consensus Strategies for Distributed Estimation Over Adaptive Networks , 2012, IEEE Transactions on Signal Processing.

[36]  Ali H. Sayed,et al.  On the Influence of Informed Agents on Learning and Adaptation Over Networks , 2012, IEEE Transactions on Signal Processing.

[37]  Yih-Fang Huang,et al.  Distributed Least Mean-Square Estimation With Partial Diffusion , 2014, IEEE Transactions on Signal Processing.

[38]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[39]  Ali H. Sayed,et al.  Adaptive regularized diffusion adaptation over multitask networks , 2015, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP).

[40]  Ali H. Sayed,et al.  Incremental Adaptive Strategies Over Distributed Networks , 2007, IEEE Transactions on Signal Processing.

[41]  Ali H. Sayed,et al.  Distributed Clustering and Learning Over Networks , 2014, IEEE Transactions on Signal Processing.