Learning Additive Noise Channels: Generalization Bounds and Algorithms

An additive noise channel is considered, in which the noise distribution is unknown and does not known to belong to any parametric family. The problem of designing a codebook and a generalized minimal distance decoder (which is parameterized by a covariance matrix) based on samples of the noise is considered. High probability generalization bounds for the error probability loss function, as well as for a hinge-type surrogate loss function are provided. A stochastic-gradient based alternating-minimization algorithm for the latter loss function is presented. Bounds on the average empirical error and generalization error are provided for a Gibbs based algorithm that gradually expurgates codewords from a large initial codebook to obtain a smaller codebook with improved error probability.

[1]  R. Zamir,et al.  Lattice Coding for Signals and Networks: A Structured Coding Approach to Quantization, Modulation and Multiuser Information Theory , 2014 .

[2]  G. David Forney,et al.  Multidimensional constellations. I. Introduction, figures of merit, and generalized cross constellations , 1989, IEEE J. Sel. Areas Commun..

[3]  Tao Jiang,et al.  Deep learning for wireless physical layer: Opportunities and challenges , 2017, China Communications.

[4]  Sreeram Kannan,et al.  Communication Algorithms via Deep Learning , 2018, ICLR.

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  Prakash Narayan,et al.  Reliable Communication Under Channel Uncertainty , 1998, IEEE Trans. Inf. Theory.

[7]  Tim Schenk,et al.  RF Imperfections in High-rate Wireless Systems: Impact and Digital Compensation , 2008 .

[8]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[9]  T. Linder.  LEARNING-THEORETIC METHODS IN VECTOR QUANTIZATION , 2002 .

[10]  Tamás Linder,et al.  Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding , 1994, IEEE Trans. Inf. Theory.

[11]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.

[12]  Sreeram Kannan,et al.  LEARN Codes: Inventing Low-Latency Codes via Recurrent Neural Networks , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[13]  Yonina C. Eldar,et al.  ViterbiNet: Symbol Detection Using a Deep Learning Based Viterbi Algorithm , 2019, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[14]  David Burshtein,et al.  Deep Learning Methods for Improved Decoding of Linear Codes , 2017, IEEE Journal of Selected Topics in Signal Processing.

[15]  Andrea J. Goldsmith,et al.  Detection Algorithms for Communication Systems Using Deep Learning , 2017, ArXiv.

[16]  Stephan ten Brink,et al.  On deep learning-based channel decoding , 2017, 2017 51st Annual Conference on Information Sciences and Systems (CISS).

[17]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[18]  Massimiliano Pontil,et al.  $K$ -Dimensional Coding Schemes in Hilbert Spaces , 2010, IEEE Transactions on Information Theory.

[19]  Tamás Linder,et al.  Empirical quantizer design in the presence of source noise or channel noise , 1997, IEEE Trans. Inf. Theory.

[20]  J.E. Mazo,et al.  Digital communications , 1985, Proceedings of the IEEE.

[21]  Robert G. Gallager,et al.  A simple derivation of the coding theorem and some applications , 1965, IEEE Trans. Inf. Theory.

[22]  Shie Mannor,et al.  The Sample Complexity of Dictionary Learning , 2010, COLT.

[23]  Mikhail Belkin,et al.  To understand deep learning we need to understand kernel learning , 2018, ICML.

[24]  R. Handel.  Probability in High Dimension , 2014 .

[25]  Jaeho Lee,et al.  Learning finite-dimensional coding schemes with nonlinear reconstruction maps , 2018, SIAM J. Math. Data Sci..

[26]  Tamás Linder,et al.  The minimax distortion redundancy in empirical quantizer design , 1997, Proceedings of IEEE International Symposium on Information Theory.