Distributed parameter estimation with exponential family statistics: Asymptotic efficiency

This paper studies the problem of distributed parameter estimation in multi-agent networks with exponential family observation statistics. Conforming to a given inter-agent communication topology, a distributed recursive estimator of the consensus-plus-innovations type is presented in which at every observation sampling epoch the network agents exchange a single round of messages with their communication neighbors and recursively update their local parameter estimates by simultaneously processing the received neighborhood data and the new information (innovation) embedded in the observation sample. Under global observability of the networked sensing model and mean connectivity of the inter-agent communication network, the proposed estimator is shown to yield consistent parameter estimates at each network agent. Furthermore, it is shown that the distributed estimator is asymptotically efficient, in that, the asymptotic covariances of the agent estimates coincide with that of the optimal centralized estimator, i.e., the inverse of the centralized Fisher information rate.

[1]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[2]  H. Vincent Poor,et al.  Distributed detection in noisy sensor networks , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[3]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[4]  Shirish Nagaraj,et al.  Set-membership filtering and a set-membership normalized LMS algorithm with an adaptive step size , 1998, IEEE Signal Processing Letters.

[5]  Soummya Kar,et al.  Distributed Parameter Estimation in Sensor Networks: Nonlinear Observation Models and Imperfect Communication , 2008, IEEE Transactions on Information Theory.

[6]  Tze Leung Lai,et al.  Asymptotic Properties of Nonlinear Least Squares Estimates in Stochastic Regression Models , 1994 .

[7]  Yih-Fang Huang,et al.  Distributed parameter estimation with selective cooperation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[8]  Srdjan S. Stankovic,et al.  Decentralized Parameter Estimation by Consensus Based Stochastic Approximation , 2007, IEEE Transactions on Automatic Control.

[9]  Soummya Kar,et al.  Asymptotically Efficient Distributed Estimation With Exponential Family Statistics , 2013, IEEE Transactions on Information Theory.

[10]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[11]  V. Fabian.  On Asymptotically Efficient Recursive Estimation , 1978 .

[12]  D. Sakrison.  Efficient recursive estimation; application to estimating the parameters of a covariance function , 1965 .

[13]  Ali H. Sayed,et al.  Diffusion Adaptation Strategies for Distributed Optimization and Learning Over Networks , 2011, IEEE Transactions on Signal Processing.

[14]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[15]  Angelia Nedic,et al.  Distributed Stochastic Subgradient Projection Algorithms for Convex Optimization , 2008, J. Optim. Theory Appl..

[16]  T. Lai,et al.  Asymptotically efficient self-tuning regulators , 1987 .

[17]  Ioannis D. Schizas,et al.  Stability analysis of the consensus-based distributed LMS algorithm , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[18]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[19]  José M. F. Moura,et al.  Distributed Detection Over Noisy Networks: Large Deviations Analysis , 2011, IEEE Transactions on Signal Processing.

[20]  Soummya Kar,et al.  Gossip Algorithms for Distributed Signal Processing , 2010, Proceedings of the IEEE.

[21]  Soummya Kar,et al.  Convergence Rate Analysis of Distributed Gossip (Linear Parameter) Estimation: Fundamental Limits and Tradeoffs , 2010, IEEE Journal of Selected Topics in Signal Processing.

[22]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[23]  H. Vincent Poor,et al.  Distributed Linear Parameter Estimation: Asymptotically Efficient Adaptive Strategies , 2011, SIAM J. Control. Optim..

[24]  H. Kushner,et al.  Asymptotic properties of distributed and communication stochastic approximation algorithms , 1987 .

[25]  C. J. Stone,et al.  Adaptive Maximum Likelihood Estimators of a Location Parameter , 1975 .

[26]  H. Vincent Poor,et al.  Bandit problems in networks: Asymptotically efficient distributed allocation rules , 2011, IEEE Conference on Decision and Control and European Control Conference.

[27]  S. Dasgupta,et al.  Asymptotically convergent modified recursive least-squares with data-dependent updating and forgetting factor , 1985 .

[28]  Angelia Nedic,et al.  Distributed and Recursive Parameter Estimation in Parametrized Linear State-Space Models , 2008, IEEE Transactions on Automatic Control.

[29]  Yih-Fang Huang,et al.  Asymptotically convergent modified recursive least-squares with data-dependent updating and forgetting factor , 1985, 1985 24th IEEE Conference on Decision and Control.