Calibration of Probabilities: The State of the Art

From the subjectivist point of view (de Finetti, 1937) a probability is a degree of belief in a proposition whose truth has not been ascertained. A probability expresses a purely internal state; there is no “right” or “correct” probability that resides somewhere “in reality” against which it can be compared. However, in many circumstances, it may become possible to verify the truth o£ falsity of the proposition to which a probability was attached. Today, we assess the probability of the proposition“it will rain tomorrow”. Tomorrow, we go outside and look at the rain gauge to see whether or not it has rained. When verification is possible, we can use it to gauge the adequacy of our probability assessments.

[1]  D. A. Seaver,et al.  Eliciting subjective probability distributions on continuous variables , 1978 .

[2]  Victor F. Weisskopf,et al.  Reactor safety study , 1976 .

[3]  G. A. Oberholzer,et al.  THE TORNADO OF JUNE 6, 1906, NEAR LA CROSSE, WIS. , 1906 .

[4]  P. A. Adams,et al.  Training in confidence-judgments. , 1958, The American journal of psychology.

[5]  A. H. Murphy,et al.  Scalar and Vector Partitions of the Probability Score: Part I. Two-State Situation , 1972 .

[6]  Ralf E. Schaefer,et al.  The assessment of subjective probability distributions: A training experiment☆ , 1973 .

[7]  Philip Williams,et al.  The Use of Confidence Factors in Forecasting , 1951 .

[8]  C. Holstein.  An Experiment in Probabilistic Weather Forecasting , 1971 .

[9]  Thomas A. Brown,et al.  Elicitation of personal probabilities and their assessment , 1975 .

[10]  Stuart Oskamp,et al.  The relationship of clinical experience and training methods to several criteria of clinical prediction. , 1962 .

[11]  Thomas A. Brown,et al.  Quantifying Uncertainty into Numerical Probabilities for the Reporting of Intelligence , 1973 .

[12]  Carl-Axel S. Staël von Holstein,et al.  Probabilistic forecasting: An experiment related to the stock market , 1972 .

[13]  Lee W. Gregg,et al.  Knowledge and cognition , 1974 .

[14]  S. Lichtenstein,et al.  Do those who know more also know more about how much they know?*1 , 1977 .

[15]  J SWETS,et al.  Decision processes in perception. , 1961, Psychological review.

[16]  J. Ivey,et al.  Ann Arbor, Michigan , 1969 .

[17]  Joan E. Sieber,et al.  Effects of decision importance on ability to generate warranted subjective uncertainty. , 1974 .

[18]  Detlof von Winterfeldt,et al.  Flat Maxima in Linear Optimization Models , 1973 .

[19]  A. H. Murphy,et al.  The Use of Credible Intervals in Temperature Forecasting: Some Experimental Results , 1977 .

[20]  A. H. Murphy,et al.  subjective probability forecasting experiments in meteorology: some preliminary results , 1974 .

[21]  J K ADAMS,et al.  A confidence scale defined in terms of expected percentages. , 1957, The American journal of psychology.

[22]  Ernest Cooke,et al.  FORECASTS AND VERIFICATIONS IN WESTERN AUSTRALIA , 2022 .

[23]  Irwin Pollack,et al.  Confidence Ratings, Message‐Reception, and the Receiver Operator Characteristic , 1957 .

[24]  P. A. Adams,et al.  Realism of confidence judgments. , 1961, Psychological review.

[25]  A. H. Murphy,et al.  credible interval temperature forecasts , 1972 .

[26]  B. Fischhoff,et al.  I knew it would happen: Remembered probabilities of once—future things , 1975 .

[27]  G. Brier.  VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY , 1950 .

[28]  J. B. Wallace,et al.  A STUDY OF THE PERFORMANCE OF SUBJECTIVE PROBABILITY ASSESSORS , 1974 .

[29]  Paul Slovic,et al.  From Shakespeare to Simon: speculations--and some evidence-- about man's ability to process information , 1972 .

[30]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[31]  Irwin Pollack,et al.  Confidence Ratings and Message Reception for Filtered Speech , 1958 .

[32]  A. H. Murphy,et al.  A Sample Skill Score for Probability Forecasts , 1974 .

[33]  Henry E. Kyburg,et al.  Studies in Subjective Probability , 1965 .

[34]  Robert Schlaifer,et al.  Computer programs for elementary decision analysis , 1971 .

[35]  A. H. Murphy,et al.  forecasters and probability forecasts: some current problems1,2 , 1971 .

[36]  Cameron R. Peterson,et al.  Odds Versus Probabilities for Categorical Events. , 1973 .

[37]  Frank R. Clarke.  Confidence Ratings, Second‐Choice Responses, and Confusion Matrices in Intelligibility Tests , 1960 .

[38]  Robert K. Weatherwax.  Virtues and Limitations of Risk Analysis. , 1975 .

[39]  B. D. Finetti.  La prévision : ses lois logiques, ses sources subjectives , 1937 .

[40]  A. H. Murphy.  A New Vector Partition of the Probability Score , 1973 .

[41]  A. H. Murphy,et al.  “Good” Probability Assessors , 1968 .

[42]  L. J. Savage.  Elicitation of Personal Probabilities and Expectations , 1971 .

[43]  Halbert E. Root,et al.  Probability Statements in Weather Forecasting , 1962 .

[44]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .