On Bounds and Closed-Form Expressions for Capacities of Discrete Memoryless Channels With Invertible Positive Matrices

While capacities of discrete memoryless channels are well studied, it is still not possible to obtain a closed-form expression for the capacity of an arbitrary discrete memoryless channel (DMC). In this paper, we study a class of DMCs whose channel matrix is an invertible positive matrix. This class of channel matrices can be used to model many real-world settings. Next, an elementary technique based on Karush-Kuhn-Tucker (KKT) conditions is used to obtain (1) a good upper bound of a discrete memoryless channel having an invertible positive channel matrix and (2) a closed-form expression for the capacity if the channel matrix satisfies certain conditions related to its singular value and its Gershgorin's disk.

[1]  Claude E. Shannon,et al.  The zero error capacity of a noisy channel , 1956, IRE Trans. Inf. Theory.

[2]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[3]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[4]  M. Fiedler,et al.  Diagonally dominant matrices , 1967 .

[5]  Saburo Muroga.  On the Capacity of a Discrete Channel, II. Mathematical Expression of the Capacity of a Noisy Channel which is Expressible by Corresponding Two Multi-state Diagrams , 1956 .

[6]  Thomas M. Cover,et al.  An achievable rate region for the broadcast channel , 1975, IEEE Trans. Inf. Theory.

[7]  Fady Alajaji,et al.  An Introduction to Single-User Information Theory , 2018 .

[8]  L. Goddard.  Information Theory , 1962, Nature.

[9]  Yaming Yu,et al.  Squeezing the Arimoto–Blahut Algorithm for Faster Convergence , 2009, IEEE Transactions on Information Theory.

[10]  Paul Cotae,et al.  Eigenvalue characterization of the capacity of discrete memoryless channels with invertible channel matrices , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[11]  Wei Yu,et al.  Blahut-Arimoto algorithms for computing channel capacity and rate-distortion with side information , 2004, International Symposium onInformation Theory, 2004. ISIT 2004. Proceedings..

[12]  Ira S. Moskowitz,et al.  Algebraic information theory for binary channels , 2010, Theor. Comput. Sci..

[13]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[14]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[15]  Thinh Nguyen,et al.  On Closed Form Capacities of Discrete Memoryless Channels , 2018, 2018 IEEE 87th Vehicular Technology Conference (VTC Spring).

[16]  Limin Zou,et al.  A lower bound for the smallest singular value , 2012 .

[17]  Amanda Sandford.  ASH , .

[18]  Stephen P. Boyd,et al.  A tutorial on geometric programming , 2007, Optimization and Engineering.

[19]  Bernd Meister,et al.  On the Capacity of a Discrete, Constant Channel , 1967, Inf. Control..

[20]  Thinh P. Nguyen,et al.  Location Assisted Coding (LAC): Embracing Interference in Free Space Optical Communications , 2015, Q2SWinet@MSWiM.

[21]  Xue-Bin Liang,et al.  An Algebraic, Analytic, and Algorithmic Investigation on the Capacity and Capacity-Achieving Input Probability Distributions of Finite-Input– Finite-Output Discrete Memoryless Channels , 2008, IEEE Transactions on Information Theory.

[22]  Armin Wittneben,et al.  Achievable Rate Regions for the Two-way Relay Channel , 2006, 2006 IEEE International Symposium on Information Theory.

[23]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[24]  Richard E. Blahut,et al.  Computation of channel capacity and rate-distortion functions , 1972, IEEE Trans. Inf. Theory.

[25]  Masakazu Jimbo,et al.  An Iteration Method for Calculating the Relative Capacity , 1979, Inf. Control..

[26]  Abbas El Gamal,et al.  Capacity theorems for the relay channel , 1979, IEEE Trans. Inf. Theory.

[27]  Charles R. Johnson.  A Gersgorin-type lower bound for the smallest singular value , 1989 .

[28]  Gerald Matz,et al.  Information geometric formulation and interpretation of accelerated Blahut-Arimoto-type algorithms , 2004, Information Theory Workshop.

[29]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[30]  Stephen P. Boyd,et al.  Geometric programming duals of channel capacity and rate distortion , 2004, IEEE Transactions on Information Theory.