Achievable Moderate Deviations Asymptotics for Streaming Compression of Correlated Sources

Motivated by streaming multi-view video coding and wireless sensor networks, we consider the problem of blockwise streaming compression of a pair of correlated sources, which we term streaming Slepian-Wolf coding. We study the moderate deviations regime in which the rate pairs of a sequence of codes converge, along a straight line, to various points on the boundary of the Slepian-Wolf region at a speed slower than the inverse square root of the blocklength <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula>, while the error probability decays subexponentially fast in <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula>. Our main result focuses on the directions of approaches to corner points of the Slepian-Wolf region. It states that for each correlated source and all corner points, there exists a non-empty subset of directions of approaches, such that the moderate deviations constant (the constant of proportionality for the subexponential decay of the error probability) is enhanced (over the non-streaming case) by at least a factor of <inline-formula> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula>, the block delay of decoding source block pairs. We specialize our main result to the setting of streaming lossless source coding and generalize this result to the setting, where we have different delay requirements for each of the two source blocks. The proof of our main result involves the use of various analytical tools and amalgamates several ideas from the recent information-theoretic streaming literature. We adapt the so-called truncated memory encoding idea from Draper and Khisti (2011) and Lee, Tan, and Khisti (2016) to ensure that the effect of error accumulation is nullified in the limit of large block lengths. We also adapt the use of the so-called minimum weighted empirical suffix entropy decoder, which was used by Draper, Chang, and Sahai (2014) to derive achievable error exponents for symbolwise streaming Slepian-Wolf coding.

[1]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[2]  Marc Levoy,et al.  High-speed videography using a dense camera array , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[3]  Sergio Verdú,et al.  Optimal Lossless Data Compression: Non-Asymptotics and Asymptotics , 2014, IEEE Transactions on Information Theory.

[4]  Hiroki Koga,et al.  Information-Spectrum Methods in Information Theory , 2002 .

[5]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[6]  Vincent Y. F. Tan,et al.  Exact Moderate Deviation Asymptotics in Streaming Data Transmission , 2016, IEEE Transactions on Information Theory.

[7]  Avideh Zakhor,et al.  Distributed video streaming over Internet , 2001, IS&T/SPIE Electronic Imaging.

[8]  Cheng Chang,et al.  Streaming source coding with delay , 2007 .

[9]  Walter W. Piegorsch,et al.  Analyzing Environmental Data , 2006 .

[10]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[11]  Stark C. Draper,et al.  Truncated tree codes for streaming data: Infinite-memory reliability using finite memory , 2011, 2011 8th International Symposium on Wireless Communication Systems.

[12]  Helen J. Wang,et al.  Distributing streaming media content using cooperative networking , 2002, NOSSDAV '02.

[13]  Vincent Yan Fu Tan,et al.  Moderate deviations for joint source-channel coding of systems with Markovian memory , 2014, 2014 IEEE International Symposium on Information Theory.

[14]  H. Vincent Poor,et al.  Channel coding: non-asymptotic fundamental limits , 2010 .

[15]  Daniel Minoli,et al.  Wireless Sensor Networks: Technology, Protocols, and Applications , 2007 .

[16]  Harikrishna Palaiyanur,et al.  The impact of causality on information-theoretic source and channel coding problems , 2011 .

[17]  Tomohiko Uyematsu,et al.  On the Wyner-Ziv Source Coding Problem with Unknown Delay , 2014, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[18]  Abbas El Gamal,et al.  Network Information Theory , 2021, 2021 IEEE 3rd International Conference on Advanced Trends in Information Theory (ATIT).

[19]  Stark C. Draper,et al.  The Streaming-DMT of Fading Channels , 2013, IEEE Transactions on Information Theory.

[20]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[21]  Aaron B. Wagner,et al.  Moderate Deviations in Channel Coding , 2012, IEEE Transactions on Information Theory.

[22]  B. Girod,et al.  Multiview Video Compression , 2007, IEEE Signal Processing Magazine.

[23]  John M. Wozencraft,et al.  Sequential decoding for reliable communication , 1957 .

[24]  Anant Sahai,et al.  Sequential decoding for lossless streaming source coding with side information , 2007, ArXiv.

[25]  Anant Sahai,et al.  The error exponent with delay for lossless source coding , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Punta del Este.

[26]  Vincent Y. F. Tan,et al.  Streaming data transmission in the moderate deviations and central limit regimes , 2015, 2016 IEEE International Symposium on Information Theory (ISIT).

[27]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[28]  Leonard J. Schulman Coding for interactive communication , 1996, IEEE Trans. Inf. Theory.

[29]  Krishna P. Gummadi,et al.  An analysis of Internet content delivery systems , 2002, OPSR.

[30]  S. Verdú,et al.  Channel dispersion and moderate deviations limits for memoryless channels , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[31]  Jun Chen,et al.  On the Redundancy-Error Tradeoff in Slepian-Wolf Coding and Channel Coding , 2007, 2007 IEEE International Symposium on Information Theory.

[32]  Prakash Ishwar,et al.  On Delayed Sequential Coding of Correlated Sources , 2011, IEEE Transactions on Information Theory.

[33]  Thomas M. Cover,et al.  A Proof of the Data Compression Theorem of Slepian and Wolf for Ergodic Sources , 1971 .

[34]  Masahito Hayashi,et al.  Second-Order Asymptotics in Fixed-Length Source Coding and Intrinsic Randomness , 2005, IEEE Transactions on Information Theory.

[35]  Masahito Hayashi,et al.  Secure multiplex coding with dependent and non-uniform multiple messages , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[36]  Anant Sahai,et al.  The price of ignorance: The impact of side-information on delay for lossless source-coding , 2007, ArXiv.

[37]  Jun Chen,et al.  A Truncated Prediction Framework for Streaming Over Erasure Channels , 2017, IEEE Transactions on Information Theory.

[38]  Nan Zhang,et al.  Delay exponent of variable-length random binning for point-to-point transmission , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[39]  Vincent Yan Fu Tan,et al.  Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities , 2014, Found. Trends Commun. Inf. Theory.

[40]  Mehul Motani,et al.  Achievable moderate deviations asymptotics for streaming Slepian-Wolf coding , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[41]  Jun Chen,et al.  On the Redundancy of Slepian–Wolf Coding , 2009, IEEE Transactions on Information Theory.

[42]  Ramji Venkataramanan,et al.  Source Coding With Feed-Forward: Rate-Distortion Theorems and Error Exponents for a General Source , 2007, IEEE Transactions on Information Theory.

[43]  Anant Sahai,et al.  Lossless Coding for Distributed Streaming Sources , 2006, IEEE Transactions on Information Theory.

[44]  Yao Wang,et al.  Multiview video sequence analysis, compression, and virtual viewpoint synthesis , 2000, IEEE Trans. Circuits Syst. Video Technol..

[45]  Ioannis Kontoyiannis,et al.  Lossless compression with moderate error probability , 2013, 2013 IEEE International Symposium on Information Theory.

[46]  Aaron B. Wagner,et al.  Moderate deviation analysis of channel coding: Discrete memoryless case , 2010, 2010 IEEE International Symposium on Information Theory.

[47]  Wen Gao,et al.  Wyner–Ziv-Based Multiview Video Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[48]  Vincent Yan Fu Tan,et al.  Asymmetric Evaluations of Erasure and Undetected Error Probabilities , 2014, IEEE Transactions on Information Theory.

[49]  Farrokh Etezadi,et al.  Streaming of Markov Sources over Burst Erasure Channels , 2015 .

[50]  Anant Sahai,et al.  Anytime information theory , 2001 .

[51]  Vincent Y. F. Tan,et al.  Moderate-deviations of lossy source coding for discrete and Gaussian sources , 2011, 2012 IEEE International Symposium on Information Theory Proceedings.

[52]  Babak Hassibi,et al.  Linear error correcting codes with anytime reliability , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[53]  Anant Sahai Why Do Block Length and Delay Behave Differently if Feedback Is Present? , 2008, IEEE Transactions on Information Theory.

[54]  Thomas H. Cormen,et al.  Introduction to algorithms [2nd ed.] , 2001 .