Observation Uncertainty in Gaussian Sensor Networks

Abstract : The term "sensor network" encompasses a wide range of engineering systems with dramatically different characteristics. We consider a specific class of sensor networks whose objective is to reconstruct a source at a central terminal. Our objective in this thesis is to quantify the asymptotic error in reconstructing the source as the number of data sources, sensors, and model complexity increases. We consider three types of estimation systems - unconstrained estimators for vector Gaussian sources that are allowed direct access to the sensor observations, estimators for discrete sources that receive information via rate constrained links from the sensors, and estimators for scalar Gaussians whose input is the output of a multiple-access channel. We first establish bounds on the optimal estimator performance of these networks using a centralized estimator with access to all of the sensor observations. We assume the observations are noisy linear functions of the source and are thus specified by a matrix. Because the asymptotic error depends only on the spectral properties of this matrix, we can use tools from matrix analysis to give bounds on the spectrum and error in terms of the entries of the matrix for a number of different scenarios. Finally, we look at the case where the matrix is partially unknown. In some cases we can estimate the matrix directly from the data and in others we must minimize the worst mismatch distortion.

[1]  M. Vetterli,et al.  Power-bandwidth-distortion scaling laws for sensor networks , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[2]  D. J. Wheeler,et al.  A Block-sorting Lossless Data Compression Algorithm , 1994 .

[3]  Toby Berger,et al.  The quadratic Gaussian CEO problem , 1997, IEEE Trans. Inf. Theory.

[4]  Anand D. Sarwate,et al.  Broadcast gossip algorithms: Design and analysis for consensus , 2008, 2008 47th IEEE Conference on Decision and Control.

[5]  Venkat Anantharam,et al.  An improved outer bound for the multiterminal source-coding problem , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[6]  M. Foucault,et al.  Discipline and Punish: The Birth of the Prison , 2020, On Violence.

[7]  U. Grenander,et al.  Toeplitz Forms And Their Applications , 1958 .

[8]  Frans M. J. Willems,et al.  The context-tree weighting method: basic properties , 1995, IEEE Trans. Inf. Theory.

[9]  A. W. M. van den Enden,et al.  Discrete Time Signal Processing , 1989 .

[10]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[11]  Tsachy Weissman,et al.  The minimax distortion redundancy in noisy source coding , 2003, IEEE Trans. Inf. Theory.

[12]  Anand D. Sarwate,et al.  Estimation from Misaligned Observations with Limited Feedback , 2005 .

[13]  Yasutada Oohama Universal coding for correlated sources with linked encoders , 1996, IEEE Trans. Inf. Theory.

[14]  Z. Bai METHODOLOGIES IN SPECTRAL ANALYSIS OF LARGE DIMENSIONAL RANDOM MATRICES , A REVIEW , 1999 .

[15]  Anand D. Sarwate,et al.  Arbitrarily dirty paper coding and applications , 2008, 2008 IEEE International Symposium on Information Theory.

[16]  Boris Tsybakov,et al.  Information transmission with additional noise , 1962, IRE Trans. Inf. Theory.

[17]  R. D'amico Discipline and Punish: The Birth of the Prison , 1978, Telos.

[18]  Anand D. Sarwate,et al.  Randomization for robust communication in networks, or Brother, can you spare a bit? , 2006 .

[19]  R. Durrett Probability: Theory and Examples , 1993 .

[20]  L. Ammann Robust Singular Value Decompositions: A New Approach to Projection Pursuit , 1993 .

[21]  Imre Csiszár,et al.  Towards a general theory of source networks , 1980, IEEE Trans. Inf. Theory.

[22]  Douglas G. Simpson Nearest-neighbor variance estimation (NNVE): Robust covariance estimation via nearest-neighbor cleaning - Comment , 2002 .

[23]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[24]  Michael Gastpar,et al.  Source-Channel Communication in Sensor Networks , 2003, IPSN.

[25]  Anand D. Sarwate,et al.  Channels with nosy "noise" , 2007, 2007 IEEE International Symposium on Information Theory.

[26]  Anant Sahai,et al.  Sequential random binning for streaming distributed source coding , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[27]  Michelle Effros,et al.  Lossless and near-lossless source coding for multiple access networks , 2003, IEEE Trans. Inf. Theory.

[28]  Yasutada Oohama,et al.  Rate-distortion theory for Gaussian multiterminal source coding systems with several side informations at the decoder , 2005, IEEE Transactions on Information Theory.

[29]  R. Dykstra,et al.  REML Estimation of Covariance Matrices with Restricted Parameter Spaces , 1995 .

[30]  Vinod M. Prabhakaran,et al.  Rate region of the quadratic Gaussian CEO problem , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[31]  Anand D. Sarwate,et al.  Fading observation alignment via feedback , 2005, IPSN 2005. Fourth International Symposium on Information Processing in Sensor Networks, 2005..

[32]  M. Gastpar,et al.  Rateless coding with partial CSI at the decoder , 2007, 2007 IEEE Information Theory Workshop.

[33]  Victor J. Yohai,et al.  The Behavior of the Stahel-Donoho Robust Multivariate Estimator , 1995 .

[34]  Michelle Effros,et al.  Universal lossless source coding with the Burrows Wheeler transform , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[35]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[36]  Jelena Kovacevic,et al.  Wavelets and Subband Coding , 2013, Prentice Hall Signal Processing Series.

[37]  Richard G. Baraniuk,et al.  Redundancy Rates of Slepian-Wolf Coding ∗ , 2004 .

[38]  I. M. Pyshik,et al.  Table of integrals, series, and products , 1965 .

[39]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[40]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[41]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[42]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[43]  Anand D. Sarwate,et al.  Randomization bounds on Gaussian arbitrarily varying channels , 2006, 2006 IEEE International Symposium on Information Theory.

[44]  Venkat Anantharam,et al.  An Infeasibility Result for the Multiterminal Source-Coding Problem , 2005, ArXiv.

[45]  Anand D. Sarwate,et al.  Using zero-rate feedback on binary additive channels with individual noise sequences , 2007, 2007 IEEE International Symposium on Information Theory.

[46]  Yasutada Oohama,et al.  The Rate-Distortion Function for the Quadratic Gaussian CEO Problem , 1998, IEEE Trans. Inf. Theory.

[47]  Anand D. Sarwate,et al.  Adversarial interference models for multiantenna cooperative systems , 2008, 2008 42nd Annual Conference on Information Sciences and Systems.

[48]  Stark C. Draper Universal Incremental Slepian-Wolf Coding , 2004 .