A General Method for Comparing Probability Assessors

A probability assessor or forecaster is a person who assigns subjective probabilities to events which will eventually occur or not occur. In this paper a general method for comparing forecasters after a finite number of trials is introduced. The general method is proven to include calculating all proper scoring rules as special cases. It also includes comparison of forecasters in all simple two-decision problems as special cases. The relationship between the general method and calibration is also explored. The general method is also translated into a method for deciding who will give better forecasts in the future