On Error Exponents and Moderate Deviations for Lossless Streaming Compression of Correlated Sources

We derive upper and lower bounds for the error exponents of lossless streaming compression of two correlated sources under the blockwise and symbolwise settings. We consider the linear scaling regime in which the delay is a scalar multiple of the number of symbol pairs of interest. We show that for rate pairs satisfying certain constraints, the upper and lower bounds for the error exponent of blockwise codes coincide. For symbolwise codes, the bounds coincide for rate pairs satisfying the aforementioned constraints and a certain condition on the symbol pairs we wish to decode---namely, that their indices are asymptotically comparable to the blocklength. We also derive moderate deviations constants for blockwise and symbolwise codes, leveraging the error exponent results, and using appropriate Taylor series expansions. In particular, for blockwise codes, we derive an information spectrum-type strong converse, giving the complete characterization of the moderate deviations constants. For symbolwise codes, under an additional requirement on the backoff from the first-order fundamental limit, we can show that the moderate deviations constants are the same as the blockwise setting.

[1]  Sergio Verdú,et al.  Optimal Lossless Data Compression: Non-Asymptotics and Asymptotics , 2014, IEEE Transactions on Information Theory.

[2]  Anant Sahai,et al.  Sequential decoding for lossless streaming source coding with side information , 2007, ArXiv.

[3]  Vincent Yan Fu Tan,et al.  Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities , 2014, Found. Trends Commun. Inf. Theory.

[4]  S. Verdú,et al.  Channel dispersion and moderate deviations limits for memoryless channels , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[5]  Jun Chen,et al.  On the Redundancy-Error Tradeoff in Slepian-Wolf Coding and Channel Coding , 2007, 2007 IEEE International Symposium on Information Theory.

[6]  Hiroki Koga,et al.  Information-Spectrum Methods in Information Theory , 2002 .

[7]  Ramji Venkataramanan,et al.  Source Coding With Feed-Forward: Rate-Distortion Theorems and Error Exponents for a General Source , 2007, IEEE Transactions on Information Theory.

[8]  Prakash Ishwar,et al.  On Delayed Sequential Coding of Correlated Sources , 2011, IEEE Transactions on Information Theory.

[9]  Avideh Zakhor,et al.  Distributed video streaming over Internet , 2001, IS&T/SPIE Electronic Imaging.

[10]  Anant Sahai,et al.  Lossless Coding for Distributed Streaming Sources , 2006, IEEE Transactions on Information Theory.

[11]  Anant Sahai,et al.  The error exponent with delay for lossless source coding , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Punta del Este.

[12]  Ioannis Kontoyiannis,et al.  Lossless compression with moderate error probability , 2013, 2013 IEEE International Symposium on Information Theory.

[13]  Aaron B. Wagner,et al.  Moderate Deviations in Channel Coding , 2012, IEEE Transactions on Information Theory.

[14]  Jun Chen,et al.  On the Redundancy of Slepian–Wolf Coding , 2009, IEEE Transactions on Information Theory.

[15]  Krishna P. Gummadi,et al.  An analysis of Internet content delivery systems , 2002, OPSR.

[16]  Vincent Y. F. Tan,et al.  On the dispersions of three network information theory problems , 2012, 2012 46th Annual Conference on Information Sciences and Systems (CISS).

[17]  Helen J. Wang,et al.  Distributing streaming media content using cooperative networking , 2002, NOSSDAV '02.

[18]  Vincent Yan Fu Tan,et al.  Moderate deviations for joint source-channel coding of systems with Markovian memory , 2014, 2014 IEEE International Symposium on Information Theory.

[19]  Te Sun Han,et al.  Second-order Slepian-Wolf coding theorems for non-mixed and mixed sources , 2012, 2013 IEEE International Symposium on Information Theory.

[20]  Cheng Chang,et al.  Streaming source coding with delay , 2007 .

[21]  Rudolf Ahlswede,et al.  Source coding with side information and a converse for degraded broadcast channels , 1975, IEEE Trans. Inf. Theory.

[22]  Tomohiko Uyematsu,et al.  On the Wyner-Ziv Source Coding Problem with Unknown Delay , 2014, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[23]  Thomas M. Cover,et al.  A Proof of the Data Compression Theorem of Slepian and Wolf for Ergodic Sources , 1971 .

[24]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[25]  Jack K. Wolf,et al.  Noiseless coding of correlated information sources , 1973, IEEE Trans. Inf. Theory.

[26]  Aaron D. Wyner,et al.  On source coding with side information at the decoder , 1975, IEEE Trans. Inf. Theory.

[27]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[28]  F. Kanaya,et al.  Coding Theorems on Correlated General Sources , 1995 .

[29]  Vincent Y. F. Tan,et al.  Moderate-deviations of lossy source coding for discrete and Gaussian sources , 2011, 2012 IEEE International Symposium on Information Theory Proceedings.

[30]  Anant Sahai Why Do Block Length and Delay Behave Differently if Feedback Is Present? , 2008, IEEE Transactions on Information Theory.

[31]  Y F TanVincent Asymptotic Estimates in Information Theory with Non-Vanishing Error Probabilities , 2014 .

[32]  Nan Zhang,et al.  Delay exponent of variable-length random binning for point-to-point transmission , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[33]  Aaron B. Wagner,et al.  Moderate deviation analysis of channel coding: Discrete memoryless case , 2010, 2010 IEEE International Symposium on Information Theory.

[34]  Masahito Hayashi,et al.  Secure multiplex coding with dependent and non-uniform multiple messages , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[35]  Anant Sahai,et al.  The price of ignorance: The impact of side-information on delay for lossless source-coding , 2007, ArXiv.