Smooth calibration, leaky forecasts, finite recall, and Nash dynamics

We propose to smooth out the calibration score, which measures how good a forecaster is, by combining nearby forecasts. While regular calibration can be guaranteed only by randomized forecasting procedures, we show that smooth calibration can be guaranteed by deterministic procedures. As a consequence, it does not matter if the forecasts are leaked, i.e., made known in advance: smooth calibration can nevertheless be guaranteed (while regular calibration cannot). Moreover, our procedure has finite recall, is stationary, and all forecasts lie on a finite grid. To construct the procedure, we deal also with the related setups of online linear regression and weak calibration. Finally, we show that smooth calibration yields uncoupled finite-memory dynamics in n-person games—“smooth calibrated learning”—in which the players play approximate Nash equilibria in almost all periods (by contrast, calibrated learning, which uses regular calibration, yields only that the time averages of play are approximate correlated equilibria).

[1]  Dean Phillips Foster,et al.  Regret Testing: Learning to Play Nash Equilibrium Without Knowing You Have an Opponent , 2006 .

[2]  David Oakes,et al.  Self-Calibrating Priors Do Not Exist , 1985 .

[3]  A. Dawid Comment: The Impossibility of Inductive Inference , 1985 .

[4]  S. Hart,et al.  Simple Adaptive Strategies: From Regret-matching To Uncoupled Dynamics , 2013 .

[5]  Wojciech Olszewski,et al.  Calibration and Expert Testing , 2015 .

[6]  Christos H. Papadimitriou,et al.  On the Complexity of the Parity Argument and Other Inefficient Proofs of Existence , 1994, J. Comput. Syst. Sci..

[7]  Dean P. Foster,et al.  A Proof of Calibration Via Blackwell's Approachability Theorem , 1999 .

[8]  Sham M. Kakade,et al.  (weak) Calibration is Computationally Hard , 2012, COLT.

[9]  S. Hart Adaptive Heuristics , 2005 .

[10]  S. Hart,et al.  A General Class of Adaptive Strategies , 1999 .

[11]  Manfred K. Warmuth,et al.  Relative Loss Bounds for On-Line Density Estimation with the Exponential Family of Distributions , 1999, Machine Learning.

[12]  Ehud Lehrer,et al.  Author's Personal Copy Games and Economic Behavior Approachability with Bounded Memory , 2022 .

[13]  Abubakr Gafar Abdalla,et al.  Probability Theory , 2017, Encyclopedia of GIS.

[14]  A. Dawid The Well-Calibrated Bayesian , 1982 .

[15]  Fabrizio Germano,et al.  Global Nash Convergence of Foster and Young's Regret Testing , 2004, Games Econ. Behav..

[16]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[17]  H. Peyton Young,et al.  Learning by trial and error , 2009, Games Econ. Behav..

[18]  Yakov Babichenko,et al.  Completely uncoupled dynamics and Nash equilibria , 2012, Games Econ. Behav..

[19]  Sham M. Kakade,et al.  Deterministic calibration and Nash equilibrium , 2004, J. Comput. Syst. Sci..

[20]  Jürgen Forster,et al.  On Relative Loss Bounds in Generalized Linear Regression , 1999, FCT.

[21]  V. Vovk Competitive On‐line Statistics , 2001 .

[22]  H. Peyton Young,et al.  Learning, hypothesis testing, and Nash equilibrium , 2003, Games Econ. Behav..

[23]  Michel Loève,et al.  Probability Theory I , 1977 .

[24]  Dean Phillips Foster Prediction in the Worst Case , 1991 .

[25]  Sham M. Kakade,et al.  Calibration via Regression , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Punta del Este.

[26]  S. Hart,et al.  A simple adaptive procedure leading to correlated equilibrium , 2000 .

[27]  R. Vohra,et al.  Calibrated Learning and Correlated Equilibrium , 1996 .

[28]  Mark J. Schervish,et al.  Self-Calibrating Priors Do Not Exist: Comment , 1985 .

[29]  Sergiu Hart,et al.  Markets, correlation, and regret-matching , 2015, Games Econ. Behav..

[30]  S. Hart,et al.  Uncoupled Dynamics Do Not Lead to Nash Equilibrium , 2003 .