Complexity of simple nonlogarithmic loss functions

The loss complexity for nonlogarithmic loss functions is defined analogously to the stochastic complexity for logarithmic loss functions such that its mean provides an achievable lower bound for estimation, the mean taken with respect to the worst case data generating distribution. The loss complexity also provides a lower bound for the worst case mean prediction error for all predictors. For the important /spl alpha/-loss functions |y-/spl circ/y|/sup /spl alpha//, where y-y/spl circ/ denotes the prediction or fitting error and /spl alpha/ is in the interval, an accurate asymptotic formula for the loss complexity is given.

[1]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[2]  Jorma Rissanen,et al.  The Minimum Description Length Principle in Coding and Modeling , 1998, IEEE Trans. Inf. Theory.

[3]  Jorma Rissanen,et al.  Strong optimality of the normalized ML models as universal codes and information in data , 2001, IEEE Trans. Inf. Theory.

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  Kenji Yamanishi,et al.  Minimax relative loss analysis for sequential prediction algorithms using parametric hypotheses , 1998, COLT' 98.

[6]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[7]  James S. Harris,et al.  Probability theory and mathematical statistics , 1998 .

[8]  Vijay Balasubramanian,et al.  Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability Distributions , 1996, Neural Computation.

[9]  Peter Gr Unwald The minimum description length principle and reasoning under uncertainty , 1998 .

[10]  W. H. Williams,et al.  Probability Theory and Mathematical Statistics , 1964 .

[11]  Kenji Yamanishi,et al.  A Decision-Theoretic Extension of Stochastic Complexity and Its Applications to Learning , 1998, IEEE Trans. Inf. Theory.

[12]  E. Hannan,et al.  Recursive estimation of autoregressions , 1989 .

[13]  Kenji Yamanishi Extended Stochastic Complexity and Minimax Relative Loss Analysis , 1999, ALT.

[14]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[15]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.