Distributed Parameter Estimation in Sensor Networks: Nonlinear Observation Models and Imperfect Communication

The paper studies distributed static parameter (vector) estimation in sensor networks with nonlinear observation models and noisy intersensor communication. It introduces separably estimable observation models that generalize the observability condition in linear centralized estimation to nonlinear distributed estimation. It studies two distributed estimation algorithms in separably estimable models, the NU (with its linear counterpart LU) and the NLU. Their update rule combines a consensus step (where each sensor updates the state by weight averaging it with its neighbors' states) and an innovation step (where each sensor processes its local current observation). This makes the three algorithms of the consensus + innovations type, very different from traditional consensus. This paper proves consistency (all sensors reach consensus almost surely and converge to the true parameter value), efficiency, and asymptotic unbiasedness. For LU and NU, it proves asymptotic normality and provides convergence rate guarantees. The three algorithms are characterized by appropriately chosen decaying weight sequences. Algorithms LU and NU are analyzed in the framework of stochastic approximation theory; algorithm NLU exhibits mixed time-scale behavior and biased perturbations, and its analysis requires a different approach that is developed in this paper.

[1]  John N. Tsitsiklis,et al.  Problems in decentralized decision making and computation , 1984 .

[2]  R. Srikant,et al.  Quantized Consensus , 2006, 2006 IEEE International Symposium on Information Theory.

[3]  Tze Leung Lai,et al.  Limiting Behavior of Weighted Sums of Independent Random Variables , 1973 .

[4]  Mehran Mesbahi,et al.  Distributed Linear Parameter Estimation in Sensor Networks based on Laplacian Dynamics Consensus Algorithm , 2006, 2006 3rd Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks.

[5]  José M. F. Moura,et al.  Distributed Detection via Gaussian Running Consensus: Large Deviations Asymptotic Analysis , 2011, IEEE Transactions on Signal Processing.

[6]  Michael William Newman,et al.  The Laplacian spectrum of graphs , 2001 .

[7]  Ali H. Sayed,et al.  Diffusion Least-Mean Squares Over Adaptive Networks: Formulation and Performance Analysis , 2008, IEEE Transactions on Signal Processing.

[8]  Soummya Kar,et al.  Sensor Networks With Random Links: Topology Design for Distributed Consensus , 2007, IEEE Transactions on Signal Processing.

[9]  Soummya Kar,et al.  Convergence Rate Analysis of Distributed Gossip (Linear Parameter) Estimation: Fundamental Limits and Tradeoffs , 2010, IEEE Journal of Selected Topics in Signal Processing.

[10]  A. Fuller,et al.  Stability of Motion , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  J. Hale,et al.  Stability of Motion. , 1964 .

[12]  R. Gray,et al.  Dithered Quantizers , 1993, Proceedings. 1991 IEEE International Symposium on Information Theory.

[13]  S. Mitter,et al.  RECURSIVE STOCHASTIC ALGORITHMS FOR GLOBAL OPTIMIZATION IN , 2022 .

[14]  Soummya Kar,et al.  Topology for Distributed Inference on Graphs , 2006, IEEE Transactions on Signal Processing.

[15]  Jonathan H. Manton,et al.  Stochastic approximation for consensus seeking: Mean square and almost sure convergence , 2007, 2007 46th IEEE Conference on Decision and Control.

[16]  Béla Bollobás,et al.  Modern Graph Theory , 2002, Graduate Texts in Mathematics.

[17]  Stephen P. Boyd,et al.  Fast linear iterations for distributed averaging , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[18]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[19]  Stephen P. Boyd,et al.  Randomized gossip algorithms , 2006, IEEE Transactions on Information Theory.

[20]  Soummya Kar,et al.  Distributed average consensus in sensor networks with quantized inter-sensor communication , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  V. Borkar Stochastic Approximation: A Dynamical Systems Viewpoint , 2008 .

[22]  Mehran Mesbahi,et al.  Agreement over random networks , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[23]  Marija D. Ilic,et al.  Dynamics and control of large electric power systems , 2000 .

[24]  William F. Stout Some Results on the Complete and Almost Sure Convergence of Linear Combinations of Independent Random Variables and Martingale Differences , 1968 .

[25]  Mikhail Borisovich Nevelʹson,et al.  Stochastic Approximation and Recursive Estimation , 1976 .

[26]  E. L. Lehmann,et al.  Theory of point estimation , 1950 .

[27]  Frank Harary,et al.  Graph Theory , 2016 .

[28]  Gerry Leversha,et al.  Foundations of modern probability (2nd edn), by Olav Kallenberg. Pp. 638. £49 (hbk). 2002. ISBN 0 387 95313 2 (Springer-Verlag). , 2004, The Mathematical Gazette.

[29]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[30]  S. Mitter,et al.  Recursive stochastic algorithms for global optimization in R d , 1991 .

[31]  Ioannis D. Schizas,et al.  Stability analysis of the consensus-based distributed LMS algorithm , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[32]  Han-Fu Chen,et al.  Identification and Stochastic Adaptive Control , 1991 .

[33]  José M. F. Moura,et al.  Distributing the Kalman Filter for Large-Scale Systems , 2007, IEEE Transactions on Signal Processing.

[34]  A. Sripad,et al.  A necessary and sufficient condition for quantization errors to be uniform and white , 1977 .

[35]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[36]  R. Olfati-Saber,et al.  Distributed Kalman Filter with Embedded Consensus Filters , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[37]  Y. S. Chow Some Convergence Theorems for Independent Random Variables , 1966 .

[38]  H. Kushner,et al.  Asymptotic properties of distributed and communication stochastic approximation algorithms , 1987 .

[39]  José M. F. Moura,et al.  Weight Optimization for Consensus Algorithms With Correlated Switching Topology , 2009, IEEE Transactions on Signal Processing.

[40]  Angelia Nedic,et al.  Distributed and Recursive Parameter Estimation in Parametrized Linear State-Space Models , 2008, IEEE Transactions on Automatic Control.

[41]  Soummya Kar,et al.  Distributed Consensus Algorithms in Sensor Networks: Quantized Data and Random Link Failures , 2007, IEEE Transactions on Signal Processing.

[42]  Soummya Kar Large scale networked dynamical systems: Distributed inference , 2010 .

[43]  John N. Tsitsiklis,et al.  On distributed averaging algorithms and quantization effects , 2007, 2008 47th IEEE Conference on Decision and Control.

[44]  J. Thorp,et al.  Real Time Voltage-Phasor Measurements for Static State Estimation , 1985, IEEE Power Engineering Review.

[45]  Soummya Kar,et al.  Distributed Kalman Filtering : Weak Consensus Under Weak Detectability , 2011 .

[46]  Alejandro Ribeiro,et al.  Consensus in Ad Hoc WSNs With Noisy Links—Part I: Distributed Estimation of Deterministic Signals , 2008, IEEE Transactions on Signal Processing.

[47]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[48]  Srdjan S. Stankovic,et al.  Decentralized Parameter Estimation by Consensus Based Stochastic Approximation , 2007, IEEE Transactions on Automatic Control.

[49]  Xuan Kong,et al.  Adaptive Signal Processing Algorithms: Stability and Performance , 1994 .

[50]  José M. F. Moura,et al.  Distributed Detection Over Noisy Networks: Large Deviations Analysis , 2011, IEEE Transactions on Signal Processing.

[51]  L. Schuchman Dither Signals and Their Effect on Quantization Noise , 1964 .

[52]  A. Scaglione,et al.  Differential Nested Lattice Encoding for Consensus Problems , 2007, 2007 6th International Symposium on Information Processing in Sensor Networks.

[53]  John N. Tsitsiklis,et al.  Convergence theories of distributed iterative processes: A survey , 1986 .

[54]  Ruggero Carli,et al.  Distributed Kalman filtering using consensus strategies , 2007, 2007 46th IEEE Conference on Decision and Control.

[55]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[56]  Michael Rabbat,et al.  Distributed Average Consensus using Probabilistic Quantization , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[57]  Ruggero Carli,et al.  Average consensus on networks with quantized communication , 2009 .

[58]  Richard M. Murray,et al.  Consensus problems in networks of agents with switching topology and time-delays , 2004, IEEE Transactions on Automatic Control.

[59]  John Vanderkooy,et al.  Quantization and Dither: A Theoretical Survey , 1992 .