Universal Simulation with a Fidelity Criterion

We consider the problem of universal simulation of memoryless sources and Markov sources, based on training sequence emitted from these sources. The objective is to maximize the conditional entropy of the simulated sequence given the training sequence, subject to a certain distance constraint between the probability distribution of the output sequence and the probability distribution of the input, training sequence. We derive a single-letter expression for the maximum conditional entropy and then propose a universal simulation scheme that asymptotically attains this maximum

[1]  P. J. Huber.  Robust Statistical Procedures , 1977 .

[2]  Sergio Verdú,et al.  Approximation theory of output statistics , 1993, IEEE Trans. Inf. Theory.

[3]  Mamoru Hoshi,et al.  Interval algorithm for random number generation , 1997, IEEE Trans. Inf. Theory.

[4]  Erik Ordentlich,et al.  Universal portfolios with side information , 1996, IEEE Trans. Inf. Theory.

[5]  Sergio Verdú,et al.  Channel simulation and coding with side information , 1994, IEEE Trans. Inf. Theory.

[6]  Robert M. Gray,et al.  Time-invariant trellis encoding of ergodic discrete-time sources with a fidelity criterion , 1977, IEEE Trans. Inf. Theory.

[7]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[8]  Neri Merhav,et al.  On universal simulation of information sources using training data , 2004, IEEE Transactions on Information Theory.

[9]  Sanjeev R. Kulkarni,et al.  Separation of random number generation and resolvability , 2000, IEEE Trans. Inf. Theory.

[10]  Gadiel Seroussi,et al.  On universal types , 2004, IEEE Transactions on Information Theory.

[11]  Neri Merhav.  Achievable key rates for universal simulation of random data with respect to a set of statistical tests , 2004, IEEE Transactions on Information Theory.

[12]  Robert M. Gray,et al.  Probability, Random Processes, And Ergodic Properties , 1987 .

[13]  Sergio Verdú,et al.  Simulation of random processes and rate-distortion theory , 1996, IEEE Trans. Inf. Theory.

[14]  Neri Merhav,et al.  Universal Delay-Limited Simulation , 2005, IEEE Transactions on Information Theory.

[15]  N. Merhav,et al.  Addendum to "On Universal Simulation of Information Sources Using Training Data , 2005, IEEE Trans. Inf. Theory.

[16]  Andrew R. Barron,et al.  Information-theoretic asymptotics of Bayes methods , 1990, IEEE Trans. Inf. Theory.

[17]  S.A. Kassam,et al.  Robust techniques for signal processing: A survey , 1985, Proceedings of the IEEE.