Deep Joint Source-Channel Coding of Images with Feedback

We consider wireless transmission of images in the presence of channel output feedback, by introducing an autoencoder-based deep joint source-channel coding (JSCC) scheme. We achieve impressive results in terms of the end-to-end reconstruction quality for fixed length transmission, and in terms of the average delay for variable length transmission. To the best of our knowledge, this is the first practical JSCC scheme that can fully exploit channel output feedback, demonstrating yet another setting in which modern machine learning techniques can enable the design of new and efficient communication methods that surpass the performance of traditional structured coding schemes.

[1]  Deniz Gündüz,et al.  Successive Refinement of Images with Deep Joint Source-Channel Coding , 2019, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[2]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[3]  Sreeram Kannan,et al.  DEEPTURBO: Deep Turbo Decoder , 2019, 2019 IEEE 20th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[4]  Claude E. Shannon,et al.  Two-way Communication Channels , 1961 .

[5]  Deniz Gündüz,et al.  Joint Source–Channel Codes for MIMO Block-Fading Channels , 2007, IEEE Transactions on Information Theory.

[6]  Sreeram Kannan,et al.  LEARN Codes: Inventing Low-Latency Codes via Recurrent Neural Networks , 2018, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[7]  David Minnen,et al.  Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Hamid Jafarkhani,et al.  Joint source-channel coding of images for channels with feedback , 1998 .

[9]  David Minnen,et al.  Joint Autoregressive and Hierarchical Priors for Learned Image Compression , 2018, NeurIPS.

[10]  David Burshtein,et al.  Deep Learning Methods for Improved Decoding of Linear Codes , 2017, IEEE Journal of Selected Topics in Signal Processing.

[11]  Aria Nosratinia,et al.  Progressive joint source-channel coding in feedback channels , 1999, Proceedings DCC'99 Data Compression Conference (Cat. No. PR00096).

[12]  Lucas Theis,et al.  Lossy Image Compression with Compressive Autoencoders , 2017, ICLR.

[13]  Valero Laparra,et al.  Density Modeling of Images using a Generalized Normalization Transformation , 2015, ICLR.

[14]  H.-S. Philip Wong,et al.  Joint Source-Channel Coding with Neural Networks for Analog Data Compression and Storage , 2018, 2018 Data Compression Conference.

[15]  Mikael Skoglund,et al.  Design and performance of VQ-based hybrid digital-analog joint source-channel codes , 2002, IEEE Trans. Inf. Theory.

[16]  Jakob Hoydis,et al.  An Introduction to Deep Learning for the Physical Layer , 2017, IEEE Transactions on Cognitive Communications and Networking.

[17]  Andrea J. Goldsmith,et al.  Deep Learning for Joint Source-Channel Coding of Text , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[18]  J. Pieter M. Schalkwijk,et al.  A coding scheme for additive noise channels with feedback-II: Band-limited signals , 1966, IEEE Trans. Inf. Theory.

[19]  Claude E. Shannon,et al.  The zero error capacity of a noisy channel , 1956, IRE Trans. Inf. Theory.

[20]  Sergio Verdú,et al.  The source-channel separation theorem revisited , 1995, IEEE Trans. Inf. Theory.

[21]  P. Viswanath,et al.  Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels , 2019, NeurIPS.

[22]  Andrea Goldsmith,et al.  Neural Network Detection of Data Sequences in Communication Systems , 2018, IEEE Transactions on Signal Processing.

[23]  Sergio Verdú,et al.  Joint source-channel coding with feedback , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[24]  Polina Bayvel,et al.  End-to-End Deep Learning of Optical Fiber Communications , 2018, Journal of Lightwave Technology.

[25]  V. Kafedziski.  Joint source channel coding of images over frequency selective fading channels with feedback using DCT and multicarrier block pulse amplitude modulation , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[26]  David Minnen,et al.  Variational image compression with a scale hyperprior , 2018, ICLR.

[27]  Mikael Skoglund,et al.  Soft Decoding for Vector Quantization Over Noisy Channels with Memory , 1999, IEEE Trans. Inf. Theory.

[28]  Touradj Ebrahimi,et al.  Perceptual Quality Study on Deep Learning Based Image Compression , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[29]  Jooyoung Lee,et al.  Context-adaptive Entropy Model for End-to-end Optimized Image Compression , 2018, ICLR.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Nariman Farvardin,et al.  On the performance and complexity of channel-optimized vector quantizers , 1991, IEEE Trans. Inf. Theory.

[32]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[33]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[34]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[35]  Andrea J. Goldsmith,et al.  Source and Channel Coding for Correlated Sources Over Multiuser Channels , 2008, IEEE Transactions on Information Theory.

[36]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[37]  Tor A. Ramstad,et al.  Shannon-kotel-nikov mappings in joint source-channel coding , 2009, IEEE Transactions on Communications.

[38]  Nam C. Phamdo,et al.  Hybrid digital-analog (HDA) joint source-channel codes for broadcasting and robust communications , 2002, IEEE Trans. Inf. Theory.

[39]  S. Butman.  Rate distortion over band-limited feedback channels , 1971 .

[40]  Sreeram Kannan,et al.  Deepcode: Feedback Codes via Deep Learning , 2018, IEEE Journal on Selected Areas in Information Theory.

[41]  Mikael Skoglund,et al.  On channel-constrained vector quantization and index assignment for discrete memoryless channels , 1999, IEEE Trans. Inf. Theory.

[42]  Deniz Gündüz,et al.  Deep Joint Source-channel Coding for Wireless Image Transmission , 2018, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[43]  Kannan Ramchandran,et al.  Robust image transmission over energy-constrained time-varying channels using multiresolution joint source-channel coding , 1998, IEEE Trans. Signal Process..

[44]  Thomas Kailath,et al.  An application of Shannon's rate-distortion theory to analog communication over feedback channels , 1967 .

[45]  David Minnen,et al.  Variable Rate Image Compression with Recurrent Neural Networks , 2015, ICLR.

[46]  Michael Gastpar,et al.  To code, or not to code: lossy source-channel communication revisited , 2003, IEEE Trans. Inf. Theory.

[47]  Valero Laparra,et al.  End-to-end Optimized Image Compression , 2016, ICLR.

[48]  Nariman Farvardin,et al.  A study of vector quantization for noisy channels , 1990, IEEE Trans. Inf. Theory.

[49]  Stephan ten Brink,et al.  Deep Learning-Based Polar Code Design , 2019, 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[50]  Leo I. Bluestein,et al.  Transmission of analog waveforms through channels with feedback (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[51]  Avideh Zakhor,et al.  Bit allocation for joint source/channel coding of scalable video , 2000, IEEE Trans. Image Process..