Hash Property and Fixed-Rate Universal Coding Theorems

The aim of this paper is to prove the fixed-rate universal coding theorems by using the notion of the hash property. These theorems are the fixed-rate lossless universal source coding theorem and the fixed-rate universal channel coding theorem. Since an ensemble of sparse matrices (with logarithmic column degree) satisfies the hash property requirement, it is proved that we can construct universal codes by using sparse matrices.

[1]  Thomas M. Cover,et al.  A Proof of the Data Compression Theorem of Slepian and Wolf for Ergodic Sources , 1971 .

[2]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[3]  David Burshtein,et al.  On the application of LDPC codes to arbitrary discrete-memoryless channels , 2003, IEEE Transactions on Information Theory.

[4]  Jon Feldman,et al.  LP decoding achieves capacity , 2005, SODA '05.

[5]  Shigeki Miyake,et al.  Construction of Universal Codes Using LDPC Matrices and Their Error Exponents , 2007, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[6]  Jun Muramatsu,et al.  A construction of channel code, joint source-channel code, and universal code for arbitrary stationary memoryless channels using sparse matrices , 2008, 2008 IEEE International Symposium on Information Theory.

[7]  Imre Csiszár.  Linear codes for sources and source networks: Error exponents, universal coding , 1982, IEEE Trans. Inf. Theory.

[8]  Jun Muramatsu,et al.  Hash Property and Coding Theorems for Sparse Matrices and Maximum-Likelihood Coding , 2008, IEEE Transactions on Information Theory.

[9]  Hiroki Koga,et al.  Source Coding Using Families of Universal Hash Functions , 2007, IEEE Transactions on Information Theory.

[10]  Jun Muramatsu,et al.  Low-density parity-check matrices for coding of correlated sources , 2003, IEEE Transactions on Information Theory.

[11]  Larry Carter,et al.  Universal Classes of Hash Functions , 1979, J. Comput. Syst. Sci..

[12]  Jun Muramatsu,et al.  Secret Key Agreement from Correlated Source Outputs Using Low Density Parity Check Matrices , 2006, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[13]  Martin J. Wainwright,et al.  LP Decoding Corrects a Constant Fraction of Errors , 2004, IEEE Transactions on Information Theory.

[14]  Martin J. Wainwright,et al.  Using linear programming to Decode Binary linear codes , 2005, IEEE Transactions on Information Theory.

[15]  Muriel Médard,et al.  Towards practical minimum-entropy universal decoding , 2005, Data Compression Conference.

[16]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[17]  William H. Kautz,et al.  Single-error-correcting codes for constant-weight data words , 1965, IEEE Trans. Inf. Theory.