On Sampling Continuous-Time AWGN Channels

For a continuous-time additive white Gaussian noise (AWGN) channel with possible feedback, it has been shown that as sampling gets infinitesimally fine, the mutual information of the associative discrete-time channels converges to that of the original continuous-time channel. We give in this paper more quantitative strengthenings of this result, which, among other implications, characterize how over-sampling approaches the true mutual information of a continuous-time Gaussian channel with bandwidth limit. The assumptions in our results are relatively mild. In particular, for the non-feedback case, compared to the Shannon-Nyquist sampling theorem, a widely used tool to connect continuous-time Gaussian channels to their discrete-time counterparts that requires the band-limitedness of the channel input, our results only require some integrability conditions on the power spectral density function of the input.

[1]  B. Øksendal.  Stochastic differential equations : an introduction with applications , 1987 .

[2]  Shunsuke Ihara,et al.  Information theory - for continuous systems , 1993 .

[3]  A. Shiryayev,et al.  Statistics of Random Processes I: General Theory , 1984 .

[4]  Jessica Fuerst,et al.  Stochastic Differential Equations And Applications , 2016 .

[5]  H. Nyquist,et al.  Certain factors affecting telegraph speed , 1924, Journal of the A.I.E.E..

[6]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[7]  Shlomo Shamai,et al.  Mutual information and minimum mean-square error in Gaussian channels , 2004, IEEE Transactions on Information Theory.

[8]  Jian Song,et al.  Extensions of the I-MMSE Relationship to Gaussian Channels With Feedback and Memory , 2014, IEEE Transactions on Information Theory.

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Xianming Liu,et al.  On Continuous-Time Gaussian Channels † , 2017, Entropy.

[11]  Robert G. Gallager,et al.  Variations on a Theme by Schalkwijk and Kailath , 2008, IEEE Transactions on Information Theory.

[12]  Tao Liu,et al.  Feedback Capacity of Stationary Gaussian Channels Further Examined , 2019, IEEE Transactions on Information Theory.

[13]  Tsz Hin.  Relative Entropy Convergence under Picard ’ s Iteration for Stochastic Differential Equations , 2018 .

[14]  J. Pieter M. Schalkwijk,et al.  A coding scheme for additive noise channels with feedback-II: Band-limited signals , 1966, IEEE Trans. Inf. Theory.

[15]  P. K. Chaturvedi,et al.  Communication Systems , 2002, IFIP — The International Federation for Information Processing.

[16]  Thomas Kailath,et al.  A coding scheme for additive noise channels with feedback-I: No bandwidth constraint , 1966, IEEE Trans. Inf. Theory.

[17]  Amiel Feinstein,et al.  Information and information stability of random variables and processes , 1964 .

[18]  Shlomo Shamai,et al.  An Elementary Proof of a Classical Information-Theoretic Formula , 2019, 2019 IEEE International Symposium on Information Theory (ISIT).

[19]  Xiongzhi Chen.  Brownian Motion and Stochastic Calculus , 2008 .

[20]  G. Parisi.  Brownian motion , 2005, Nature.

[21]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[22]  Young-Han Kim,et al.  Feedback Capacity of Stationary Gaussian Channels , 2006, 2006 IEEE International Symposium on Information Theory.