Memoryless Sequences for Differentiable Losses

One way to define the “randomness” of a fixed individual sequence is to ask how hard it is to predict. When prediction error is measured via squared loss, it has been established that memoryless sequences (which are, in a precise sense, hard to predict) have some of the stochastic attributes of truly random sequences. In this paper, we ask how changing the loss function used changes the set of memoryless sequences, and in particular, the stochastic attributes they possess. We answer this question for differentiable convex loss functions using tools from property elicitation, showing that the property elicited by the loss determines the stochastic attributes of the corresponding memoryless sequences. We apply our results to price calibration in prediction markets.

[1]  Nicolas S. Lambert.  Elicitation and Evaluation of Statistical Forecasts , 2010 .

[2]  V. V'yugin,et al.  Effective Convergence in Probability and an Ergodic Theorem forIndividual Random Sequences , 1998 .

[3]  Ian A. Kash,et al.  Vector-Valued Property Elicitation , 2015, COLT.

[4]  J. Wolfers,et al.  Prediction Markets , 2003 .

[5]  Andrew B. Nobel,et al.  Some stochastic properties of memoryless individual sequences , 2004, IEEE Transactions on Information Theory.

[6]  Vladimir Vovk.  Probability theory for the Brier game , 2001, Theor. Comput. Sci..

[7]  Arpit Agarwal,et al.  On Consistent Surrogate Risk Minimization and Property Elicitation , 2015, COLT.

[8]  V. Uspenskii,et al.  Can an individual sequence of zeros and ones be random? Russian Math , 1990 .

[9]  Siyu Zhang,et al.  Elicitation and Identification of Properties , 2014, COLT.

[10]  J McCarthy,et al.  MEASURES OF THE VALUE OF INFORMATION. , 1956, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Kent Osband,et al.  Providing incentives for better cost forecasting , 1985 .

[12]  G. Lugosi,et al.  On Prediction of Individual Sequences , 1998 .

[13]  Yoav Shoham,et al.  Eliciting properties of probability distributions , 2008, EC '08.

[14]  T. Gneiting.  Making and Evaluating Point Forecasts , 2009, 0912.0902.

[15]  Robin Hanson,et al.  Combinatorial Information Market Design , 2003, Inf. Syst. Frontiers.

[16]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[17]  Mark D. Reid,et al.  Composite Binary Losses , 2009, J. Mach. Learn. Res..

[18]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[19]  Per Martin-Löf,et al.  The Definition of Random Sequences , 1966, Inf. Control..

[20]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[21]  Yoav Shoham,et al.  Eliciting truthful answers to multiple-choice questions , 2009, EC '09.

[22]  A. Kolmogorov.  Three approaches to the quantitative definition of information , 1968 .

[23]  Mark D. Reid,et al.  Composite Multiclass Losses , 2011, J. Mach. Learn. Res..

[24]  Jennifer Wortman Vaughan,et al.  Efficient Market Making via Convex Optimization, and a Connection to Online Learning , 2013, TEAC.

[25]  G. Shafer,et al.  Probability and Finance: It's Only a Game! , 2001 .

[26]  Jacob D. Abernethy,et al.  A Characterization of Scoring Rules for Linear Properties , 2012, COLT.

[27]  L. J. Savage.  Elicitation of Personal Probabilities and Expectations , 1971 .

[28]  Dean P. Foster,et al.  Regret in the On-Line Decision Problem , 1999 .

[29]  David Haussler,et al.  Sequential Prediction of Individual Sequences Under General Loss Functions , 1998, IEEE Trans. Inf. Theory.

[30]  Bo Waggoner,et al.  An Axiomatic Study of Scoring Rule Markets , 2017, ITCS.