Accuracy and Forecast Standard Error of Prediction Markets

Prediction markets” are designed specifically to forecast events. Though such markets have been conduced for more than a decade, to date there is no analysis of their long-run predictive properties. We provide the first systematic evidence on the long-run predictive power of these markets by studying ex post accuracy and means of measuring ex ante forecast standard errors. Ex post, prediction markets prove accurate at long and short forecasting horizons, in absolute terms and relative to natural alternative forecasts. We use efficient markets theory and some special properties of the markets to develop forecast standard errors. Both time series and inter-market pricing relationships suggest that markets generate efficient random walks in prices. Thus, random walk projections generate reasonable confidence intervals. These confidence intervals differ dramatically from margins of error quoted in polls. We argue this is reasonable because polls do not attempt to, nor can they be expected to, measure the degree of uncertainty about the eventual election outcome conditional on their own results. In contrast, the markets incorporate this uncertainty by design. Accuracy and Forecast Standard Error of Prediction Markets “Prediction markets” are designed and conducted for the primary purpose of aggregating information so that market prices forecast future events. These markets differ from typical, naturally occurring markets in their primary role as a forecasting tool instead of a resource allocation mechanism. For example, since 1988, faculty at the Henry B. Tippie College of Business at the University of Iowa have been running markets through the Iowa Electronic Markets (IEM) project that are designed to predict election outcomes. These represent the longest running set of prediction markets known to us. They have proven efficient in forecasting the evening and week before elections. However, no analysis of their long-run forecasting power has been conducted. Here, we analyze these markets to show how prediction markets in general can serve as efficient mechanisms for aggregating information and forecasting events that can prove difficult for traditional forecasting methods. We put special focus on longer-run properties. Existing evidence (e.g., Berg, Forsythe, Nelson and Rietz, 2003, and references cited therein) shows excellent ex-post predictive accuracy for election prediction markets in the very short run (i.e., one-day-ahead forecasts using election eve prices). While this is an interesting and important result, it does not address the critical question of whether prediction markets can serve as effective long-run forecasting tools (weeks or months in advance). Here, we present the first systematic analysis of election market data on two additional properties that are important for evaluating their long-run efficacy. The first property we study is the longer-run predictive accuracy of markets relative to their natural competitors: polls. This analysis provides the first documented evidence that prediction markets are considerably more accurate long-run forecasting tools than polls across elections and across long periods of time preceding elections (instead of just on election-eve). The second property we study is the forecast standard error of market predictions. This allows us to have a (previously unavailable) measure of confidence in ex-ante market predictions. We study three means of measuring forecast Since 1993, these markets have expanded to predict many other types of events including other political outcomes, financial and accounting outcomes for companies, national and international economic phenomena, box office receipts for movies, etc. 2 standard errors. First, we show the difficulty in applying a previously developed structural model designed to explain short-run, ex post accuracy to out-of-sample data. Second, we show that the time series of forecasts from our prediction markets are consistent with efficient market random walks. From this, one can construct forecast standard errors. Third, we show that an efficient inter-market pricing relationship can be exploited to the same end. These estimated forecast standard errors appear somewhat larger, but are not significantly different from the random walk approach. We suggest that both should be used to get a reasonable estimate of forecast standard errors and confidence intervals for prediction markets. I. Prediction Markets Since Hayek (1945), economists have recognized that markets have a dual role. They allocate resources and, through the process of price discovery, they aggregate information about the values of these resources. The information aggregation role of some markets seems particularly apparent. For example, corporations cite the value of their stock as the consensus judgment of their owners about the value of the corporation’s activities. Increasingly, corporations reward managers based on this value measure. Futures and options markets aggregate information about the anticipated future values of stocks and commodities. If it is true that futures prices are the best predictors of actual future spot prices (as the “expectations hypothesis” asserts), then futures prices constitute forecasts. For example, Krueger and Kuttner (1996) discuss how the Federal Funds futures contract can be used to predict future Federal Funds rates and, hence, future Federal Reserve target rates. In most markets, if prediction uses arise, they do so as a secondary information aggregation role. However, some recent markets have been designed specifically to exploit their information aggregation Debate over the ability of futures markets to forecast future prices extends back to Keynes (1930) and Hicks (1946). Many of the arguments result from the secondary nature of information aggregation in these markets. The early “normal backwardization” versus “contago effect” arguments were based on relative power of speculators and hedgers. Today, the idea that “risk neutral” probabilities used to price futures and options differ from the “true” underlying probabilities result from relative levels of hedging demand in the markets. While the IEM markets discussed below may be subject to price deviations due to hedging activities, the narrow scope of the IEM markets, the small size of investments and analysis of individual traders (e.g., Forsythe, Nelson, Neumann and Wright, 1992, and Forsythe, Rietz and Ross, 1999) all lead us to conclude that hedging activities do not affect IEM prices significantly.