An Upgrading Algorithm With Optimal Power Law

Consider a channel <inline-formula> <tex-math notation="LaTeX">$W$ </tex-math></inline-formula> along with a given input distribution <inline-formula> <tex-math notation="LaTeX">$P_{X}$ </tex-math></inline-formula>. In certain settings, such as in the construction of polar codes, the output alphabet of <inline-formula> <tex-math notation="LaTeX">$W$ </tex-math></inline-formula> is ‘too large’, and hence we replace <inline-formula> <tex-math notation="LaTeX">$W$ </tex-math></inline-formula> by a channel <inline-formula> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> having a smaller output alphabet. We say that <inline-formula> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> is upgraded with respect to <inline-formula> <tex-math notation="LaTeX">$W$ </tex-math></inline-formula> if <inline-formula> <tex-math notation="LaTeX">$W$ </tex-math></inline-formula> is obtained from <inline-formula> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> by processing its output. In this case, the mutual information <inline-formula> <tex-math notation="LaTeX">$I(P_{X},W)$ </tex-math></inline-formula> between the input and output of <inline-formula> <tex-math notation="LaTeX">$W$ </tex-math></inline-formula> is upper-bounded by the mutual information <inline-formula> <tex-math notation="LaTeX">$I(P_{X},Q)$ </tex-math></inline-formula> between the input and output of <inline-formula> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula>. In this paper, we present an algorithm that produces an upgraded channel <inline-formula> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> from <inline-formula> <tex-math notation="LaTeX">$W$ </tex-math></inline-formula>, as a function of <inline-formula> <tex-math notation="LaTeX">$P_{X}$ </tex-math></inline-formula> and the required output alphabet size of <inline-formula> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula>, denoted <inline-formula> <tex-math notation="LaTeX">$L$ </tex-math></inline-formula>. We show that the difference in mutual informations is ‘small’. Namely, it is <inline-formula> <tex-math notation="LaTeX">$O(L^{-2/(| \mathcal {X}|-1)})$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$| \mathcal {X}|$ </tex-math></inline-formula> is the size of the input alphabet. This power law of <inline-formula> <tex-math notation="LaTeX">$L$ </tex-math></inline-formula> is optimal. We complement our analysis with numerical experiments which show that the developed algorithm improves upon the existing state-of-the-art algorithms also in non-asymptotic setups.

[1]  Alexander Vardy,et al.  Channel upgrading for semantically-secure encryption on wiretap channels , 2013, 2013 IEEE International Symposium on Information Theory.

[2]  Brian M. Kurkoski,et al.  Quantization of Binary-Input Discrete Memoryless Channels , 2011, IEEE Transactions on Information Theory.

[3]  Junya Honda,et al.  Polar coding without alphabet extension for asymmetric channels , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[4]  Erdal Arikan,et al.  Channel Polarization: A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels , 2008, IEEE Transactions on Information Theory.

[5]  Rüdiger L. Urbanke,et al.  Polar Codes are Optimal for Lossy Source Coding , 2009, IEEE Transactions on Information Theory.

[6]  Emre Telatar,et al.  On the construction of polar codes , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[7]  Amos Lapidoth,et al.  At Low SNR, Asymmetric Quantizers are Better , 2012, IEEE Transactions on Information Theory.

[8]  Junya Honda,et al.  Polar Coding Without Alphabet Extension for Asymmetric Models , 2013, IEEE Transactions on Information Theory.

[9]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[10]  Mikael Skoglund,et al.  Nested Polar Codes for Wiretap and Relay Channels , 2010, IEEE Communications Letters.

[11]  Ido Tal.  On the Construction of Polar Codes for Channels With Moderate Input Alphabet Sizes , 2017, IEEE Transactions on Information Theory.

[12]  Alexander Vardy,et al.  Constructing polar codes for non-binary alphabets and MACs , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[13]  Ido Tal,et al.  An Upgrading Algorithm with Optimal Power Law , 2020 .

[14]  Rudolf Mathar,et al.  Optimal one-bit quantizers are asymmetric for additive uniform noise , 2017, 2017 Information Theory and Applications Workshop (ITA).

[15]  Ken-ichi Iwata,et al.  Optimal quantization of B-DMCs maximizing α-mutual information with monge property , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[16]  Sergio Verdú,et al.  $f$ -Divergence Inequalities , 2015, IEEE Transactions on Information Theory.

[17]  Bobak Nazer,et al.  Information-distilling quantizers , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[18]  Naftali Tishby,et al.  Agglomerative Information Bottleneck , 1999, NIPS.

[19]  Ryuhei Mori,et al.  Performance and construction of polar codes on symmetric binary-input memoryless channels , 2009, 2009 IEEE International Symposium on Information Theory.

[20]  Alexander Vardy,et al.  Achieving the Secrecy Capacity of Wiretap Channels Using Polar Codes , 2010, IEEE Transactions on Information Theory.

[21]  V. D. Pietra,et al.  Minimum Impurity Partitions , 1992 .

[22]  Rüdiger L. Urbanke,et al.  Polar Codes for Channel and Source Coding , 2009, ArXiv.

[23]  Arman Fazeli,et al.  Polar Codes for the Deletion Channel: Weak and Strong Polarization , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[24]  Ido Tal,et al.  Channel Upgradation for Non-Binary Input Alphabets and MACs , 2017, IEEE Transactions on Information Theory.

[25]  Ken-ichi Iwata,et al.  Quantizer design for outputs of binary-input discrete memoryless channels using SMAWK algorithm , 2014, 2014 IEEE International Symposium on Information Theory.

[26]  Shlomo Shamai,et al.  Secrecy-Achieving Polar-Coding for Binary-Input Memoryless Symmetric Wire-Tap Channels , 2010, ArXiv.

[27]  Alexander Vardy,et al.  How to Construct Polar Codes , 2011, IEEE Transactions on Information Theory.

[28]  Ido Tal,et al.  Fast Polarization for Processes with Memory , 2017, 2018 IEEE International Symposium on Information Theory (ISIT).

[29]  Rongke Liu,et al.  Construction of polar codes for channels with memory , 2015, 2015 IEEE Information Theory Workshop - Fall (ITW).

[30]  Ido Tal,et al.  Universal Polarization for Processes with Memory , 2018, 2019 IEEE International Symposium on Information Theory (ISIT).

[31]  P. Chou,et al.  Minimum conditional entropy context quantization , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[32]  Ido Tal,et al.  Polar coding for processes with memory , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[33]  Onur Ozan Koyluoglu,et al.  Polar coding for secure transmission and key agreement , 2010, 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications.

[34]  Joseph M. Renes,et al.  Efficient One-Way Secret-Key Agreement and Private Channel Coding via Polarization , 2013, ASIACRYPT.

[35]  Brian M. Kurkoski,et al.  Low-complexity quantization of discrete memoryless channels , 2016, 2016 International Symposium on Information Theory and Its Applications (ISITA).

[36]  Alexander Barg,et al.  Construction of polar codes for arbitrary discrete memoryless channels , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[37]  Ido Tal,et al.  Greedy-merge degrading has optimal power-law , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).