General strong polarization

Arikan’s exciting discovery of polar codes has provided an altogether new way to efficiently achieve Shannon capacity. Given a (constant-sized) invertible matrix M, a family of polar codes can be associated with this matrix and its ability to approach capacity follows from the polarization of an associated [0,1]-bounded martingale, namely its convergence in the limit to either 0 or 1 with probability 1. Arikan showed appropriate polarization of the martingale associated with the matrix G2 = ( [complex formula not displayed] ) to get capacity achieving codes. His analysis was later extended to all matrices M which satisfy an obvious necessary condition for polarization. While Arikan’s theorem does not guarantee that the codes achieve capacity at small blocklengths (specifically in length which is a polynomial in 1/є where є is the difference between the capacity of a channel and the rate of the code), it turns out that a “strong” analysis of the polarization of the underlying martingale would lead to such constructions. Indeed for the martingale associated with G2 such a strong polarization was shown in two independent works ([Guruswami and Xia, IEEE IT ’15] and [Hassani et al., IEEE IT’14]), thereby resolving a major theoretical challenge associated with the efficient attainment of Shannon capacity. In this work we extend the result above to cover martingales associated with all matrices that satisfy the necessary condition for (weak) polarization. In addition to being vastly more general, our proofs of strong polarization are (in our view) also much simpler and modular. Key to our proof is a notion of local polarization that only depends on the evolution of the martingale in a single time step. We show that local polarization always implies strong polarization. We then apply relatively simple reasoning about conditional entropies to prove local polarization in very general settings. Specifically, our result shows strong polarization over all prime fields and leads to efficient capacity-achieving source codes for compressing arbitrary i.i.d. sources, and capacity-achieving channel codes for arbitrary symmetric memoryless channels.

[1]  Alexander Vardy,et al.  How to Construct Polar Codes , 2011, IEEE Transactions on Information Theory.

[2]  Venkatesan Guruswami,et al.  An Entropy Sumset Inequality and Polynomially Fast Convergence to Shannon Capacity Over All Alphabets , 2014, CCC.

[3]  Lele Wang,et al.  Polar coding for interference networks , 2014, 2014 IEEE International Symposium on Information Theory.

[4]  Michael Gastpar,et al.  Polar codes for broadcast channels , 2013, ISIT.

[5]  P. Huddleston Spinal Injuries After Improvised Explosive Device Incidents: Implications for Tactical Combat Casualty Care , 2012 .

[6]  Venkatesan Guruswami,et al.  Polar Codes: Speed of Polarization and Polynomial Gap to Capacity , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[7]  Rüdiger L. Urbanke,et al.  Polar Codes: Characterization of Exponent, Bounds, and Constructions , 2010, IEEE Transactions on Information Theory.

[8]  Rüdiger L. Urbanke,et al.  Unified scaling of polar codes: Error exponent, scaling exponent, moderate deviations, and error floors , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[9]  Alexander Vardy,et al.  Achieving the Secrecy Capacity of Wiretap Channels Using Polar Codes , 2010, IEEE Transactions on Information Theory.

[10]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[11]  Rüdiger L. Urbanke,et al.  Finite-Length Scaling for Polar Codes , 2013, IEEE Transactions on Information Theory.

[12]  Vincent Y. F. Tan,et al.  Polar Codes , 2016 .

[13]  Emre Telatar,et al.  Polar Codes for the m-User MAC , 2010 .

[14]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[15]  Erdal Arikan,et al.  Channel Polarization: A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels , 2008, IEEE Transactions on Information Theory.

[16]  Amiel Feinstein,et al.  Information and information stability of random variables and processes , 1964 .

[17]  Emre Telatar,et al.  Polar Codes for the Two-User Multiple-Access Channel , 2010, IEEE Transactions on Information Theory.

[18]  Eren Sasoglu,et al.  Polarization and Polar Codes , 2012, Found. Trends Commun. Inf. Theory.

[19]  Emre Telatar,et al.  On the rate of channel polarization , 2008, 2009 IEEE International Symposium on Information Theory.

[20]  Rüdiger L. Urbanke,et al.  Near-optimal finite-length scaling for polar codes over large alphabets , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[21]  R. Durrett Probability: Theory and Examples , 1993 .

[22]  Furkan Ercan,et al.  On error-correction performance and implementation of polar code list decoders for 5G , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  R. Urbanke,et al.  Polar codes for Slepian-Wolf, Wyner-Ziv, and Gelfand-Pinsker , 2010, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[24]  Toshiyuki Tanaka,et al.  Source and Channel Polarization Over Finite Fields and Reed–Solomon Matrices , 2012, IEEE Transactions on Information Theory.

[25]  Emre Telatar,et al.  Polar Codes for the $m$-User Multiple Access Channel , 2012, IEEE Transactions on Information Theory.

[26]  Arman Fazeli,et al.  Binary Linear Codes with Optimal Scaling and Quasi-Linear Complexity , 2017, ArXiv.