Universal sequential learning and decision from individual data sequences

Sequential learning and decision algorithms are investigated, with various application areas, under a family of additive loss functions for individual data sequences. Simple universal sequential schemes are known, under certain conditions, to approach optimality uniformly as fast as n-1logn, where n is the sample size. For the case of finite-alphabet observations, the class of schemes that can be implemented by finite-state machines (FSM's), is studied. It is shown that Markovian machines with sufficiently long memory exist that are asymptotically nearly as good as any given FSM (deterministic or randomized) for the purpose of sequential decision. For the continuous-valued observation case, a useful class of parametric schemes is discussed with special attention to the recursive least squares (RLS) algorithm.

[1]  H. Robbins,et al.  Asymptotic Solutions of the Compound Decision Problem for Two Completely Specified Distributions , 1955 .

[2]  D. Blackwell An analog of the minimax theorem for vector payoffs. , 1956 .

[3]  Arthur Gill,et al.  Introduction to the theory of finite-state machines , 1962 .

[4]  E. Samuel Asymptotic Solutions of the Sequential Compound Decision Problem , 1963 .

[5]  E. Samuel Convergence of the Losses of Certain Decision Rules for the Sequential Compound Decision Problem , 1964 .

[6]  D. D. Swain Bounds and rates of convergence for the extended compound estimation problem in the sequence case. , 1965 .

[7]  J. V. Ryzin,et al.  The Sequential Compound Decision Problem with $m \times n$ Finite Loss Matrix , 1966 .

[8]  M. Johns Two-action compound decision problems , 1967 .

[9]  D. Gilliland Sequential Compound Estimation , 1968 .

[10]  J. Hannan,et al.  On an Extended Compound Decision Problem , 1969 .

[11]  T. Cover,et al.  On Memory Saved by Randomization , 1971 .

[12]  Martin E. Hellman,et al.  The effects of randomization on finite-memory decision schemes , 1972, IEEE Trans. Inf. Theory.

[13]  Dennis Crippen Gilliland,et al.  Asymptotic risk stability resulting from play against the past in a sequence of decision problems , 1972, IEEE Trans. Inf. Theory.

[14]  Bruno O. Shubert,et al.  Finite-memory classification of Bernoulli sequences using reference samples (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[15]  J. Makhoul,et al.  Linear prediction: A tutorial review , 1975, Proceedings of the IEEE.

[16]  Thomas M. Cover,et al.  Compound Bayes Predictors for Sequences with Apparent Markov Structure , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[18]  Dennis Crippen Gilliland,et al.  On continuity of the Bayes response (Corresp.) , 1978, IEEE Trans. Inf. Theory.

[19]  Y. Nogami Thek-extended set-compound estimation problem in a nonregular family of distrubutions over [θ, θ+1) , 1979 .

[20]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[21]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[22]  S. Vardeman Admissible solutions of k-extended finite state set and sequence compound decision problems , 1980 .

[23]  Raphail E. Krichevsky,et al.  The performance of universal encoding , 1981, IEEE Trans. Inf. Theory.

[24]  Lee D. Davisson,et al.  Minimax noiseless universal coding for Markov sources , 1983, IEEE Trans. Inf. Theory.

[25]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[26]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[27]  Glen G. Langdon,et al.  An Introduction to Arithmetic Coding , 1984, IBM J. Res. Dev..

[28]  H. Robbins Asymptotically Subminimax Solutions of Compound Statistical Decision Problems , 1985 .

[29]  K. H. Barratt Digital Coding of Waveforms , 1985 .

[30]  Frank Thomson Leighton,et al.  Estimating a probability using finite memory , 1986, IEEE Trans. Inf. Theory.

[31]  Peter No,et al.  Digital Coding of Waveforms , 1986 .

[32]  J. Rissanen Stochastic Complexity and Modeling , 1986 .

[33]  Robert M. Gray,et al.  Probability, Random Processes, And Ergodic Properties , 1987 .

[34]  T. Cover,et al.  Asymptotic optimality and asymptotic equipartition properties of log-optimum investment , 1988 .

[35]  J. Ziv Compression, tests for randomness and estimating the statistical model of an individual sequence , 1990 .

[36]  Meir Feder,et al.  Gambling using a finite state machine , 1991, IEEE Trans. Inf. Theory.

[37]  Marcelo Weinberger,et al.  Upper Bounds On The Probability Of Sequences Emitted By Finite-state Sources And On The Redundancy Of The Lempel-Ziv Algorithm , 1991, Proceedings. 1991 IEEE International Symposium on Information Theory.

[38]  B. M. Fulk MATH , 1992 .

[39]  Marcelo J. Weinberger,et al.  Upper bounds on the probability of sequences emitted by finite-state sources and on the redundancy of the Lempel-Ziv algorithm , 1992, IEEE Trans. Inf. Theory.

[40]  P. Algoet UNIVERSAL SCHEMES FOR PREDICTION, GAMBLING AND PORTFOLIO SELECTION' , 1992 .

[41]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[42]  Neri Merhav,et al.  Universal schemes for sequential decision from individual data sequences , 1993, IEEE Trans. Inf. Theory.