Integrating prediction market and Delphi methodology into a foresight support system — Insights from an online game

Abstract In this paper, we introduce a novel approach, an electronic combination of a prediction market and Delphi methodology, to create a foresight support system (FSS). While the Delphi method has been a widely accepted foresight technique, prediction markets have been a very recent innovation to the existing decision and foresight support systems. Though, traditional prediction markets have been quite successful recently, our extension to the traditional prediction market methodology allows us to extract more valuable market information than any other prediction market since our approach provides not only a market forecast but also delivers an entire forecast distribution. The forecast distribution is generated by the aggregation of another unique characteristic of the suggested market structure namely that financial market professionals have to submit an interval forecast rather than point forecast. Based on our analysis, we demonstrate that our market is weak-form efficient and hence contains all publicly available information. In terms of forecasting accuracy, we conclude that the precision of our market improved over time and overall outperformed its benchmark.

[1]  Thomas A. Rietz,et al.  Results from a Dozen Years of Election Futures Markets Research , 2008 .

[2]  Gerrit van Bruggen,et al.  Prediction Markets as institutional forecasting support systems , 2010, Decis. Support Syst..

[3]  Ilkka Niiniluoto,et al.  Futures studies: science or art? , 2001 .

[4]  Heiko A. von der Gracht,et al.  Consensus measurement in Delphi studies , 2012 .

[5]  Margaret Faulkner,et al.  Are experts better than potential users in predicting the uptake of an innovation? Extending the use of the Juster Scale , 2009 .

[6]  Paul Gomme Iowa electronic markets , 2003 .

[7]  Gregor Brüggelambert,et al.  Information and efficiency in political stock markets: using computerized markets to predict election results , 2004 .

[8]  Sanford J. Grossman On the Impossibility of Informationally Efficient Markets , 1980 .

[9]  Koleman Strumpf,et al.  Manipulating political stock markets: A field experiment and a century of observational data , 2006 .

[10]  Stefan Luckner,et al.  STOCCER - A Forecasting Market for the FIFA World Cup 2006 , 2005 .

[11]  Richard P. Larrick,et al.  Intuitions About Combining Opinions: Misappreciation of the Averaging Principle , 2006, Manag. Sci..

[12]  Justin Wolfers,et al.  Using Prediction Markets to Track Information Flows: Evidence from Google , 2009, AMMA.

[13]  Martin Werding,et al.  'Passauer Wahlborse': Information Processing in a Political Market Experiment , 1996 .

[14]  David M. Pennock,et al.  The Real Power of Artificial Markets , 2001, Science.

[15]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Andreas Graefe,et al.  Are Prediction Markets More Accurate than Simple Surveys , 2010 .

[17]  Heinrich Arnold,et al.  IT Tools for Foresight: The Integrated Insight and Response System of Deutsche Telekom Innovation Laboratories , 2013 .

[18]  David M. Pennock,et al.  Extracting collective probabilistic forecasts from web games , 2001, KDD '01.

[19]  P. Bohm,et al.  Political Stock Markets and Unreliable Polls , 1999 .

[20]  J. Eliashberg,et al.  Demand and Supply Dynamics for Sequentially Released Products in International Markets: The Case of Motion Pictures , 2003 .

[21]  Ilan Yaniv,et al.  Overconfidence in interval estimates: What does expertise buy you? , 2008 .

[22]  James H. Fowler,et al.  Elections and Markets: The Effect of Partisanship, Policy Risk, and Electoral Margins on the Economy , 2006, The Journal of Politics.

[23]  Andrew Leigh,et al.  What Do Financial Markets Think of War in Iraq? , 2003 .

[24]  Ben Jacobsen,et al.  (In)accuracy of a European political stockmarket : The influence of common value structures , 2000 .

[25]  Heiko A. von der Gracht,et al.  The influence of information and communication technology (ICT) on future foresight processes — Results from a Delphi survey , 2014 .

[26]  Yves Pigneur,et al.  Technology Foresight for IT Investment: Multi-Criteria Decision-Making versus Prediction Markets , 2007 .

[27]  S. Bikhchandani,et al.  You have printed the following article : A Theory of Fads , Fashion , Custom , and Cultural Change as Informational Cascades , 2007 .

[28]  Anders Krogh,et al.  Neural Network Ensembles, Cross Validation, and Active Learning , 1994, NIPS.

[29]  R. L. Winkler,et al.  Averages of Forecasts: Some Empirical Results , 1983 .

[30]  Robert Forsythe,et al.  Markets as Predictors of Election Outcomes: Campaign Events and Judgement Bias in the 1993 UBC Election Stock Market , 1998 .

[31]  Thomas A. Rietz,et al.  What Makes Markets Predict Well? Evidence from the Iowa Electronic Markets , 1997 .

[32]  P. Sørensen,et al.  OUTCOME MANIPULATION IN CORPORATE PREDICTION MARKETS , 2007 .

[33]  Charles R. Plott,et al.  Information Aggregation Mechanisms: Concept, Design and Implementation for a Sales Forecasting Problem , 2002 .

[34]  Christof Weinhardt,et al.  A Prediction Market for Macro-Economic Variables , 2011, 2011 44th Hawaii International Conference on System Sciences.

[35]  Anita Elberse,et al.  The Power of Stars: Do Star Actors Drive the Success of Movies? , 2007 .

[36]  Thomas A. Rietz,et al.  Information Systems Frontiers 5:1, 79–93, 2003 C ○ 2003 Kluwer Academic Publishers. Manufactured in The Netherlands. Prediction Markets as Decision Support Systems , 2022 .

[37]  Murray Turoff,et al.  The Delphi Method: Techniques and Applications , 1976 .

[38]  Ilias P. Tatsiopoulos,et al.  PREDICTION MARKETS: AN EXTENDED LITERATURE REVIEW , 2007 .

[39]  Jose L. Salmeron,et al.  Scope and design issues in foresight support systems , 2011 .

[40]  Thomas A. Rietz,et al.  Wishes, expectations and actions: a survey on price formation in election stock markets , 1999 .

[41]  David I. Harvey,et al.  Analysis of a panel of UK macroeconomic forecasts , 2001 .

[42]  Kurt Matzler,et al.  Predicting New Product Success with Prediction Markets in Online Communities , 2013 .

[43]  N. Dalkey,et al.  An Experimental Application of the Delphi Method to the Use of Experts , 1963 .

[44]  A. Banerjee,et al.  A Simple Model of Herd Behavior , 1992 .

[45]  S. Page Prologue to The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies , 2007 .

[46]  Thomas S. Gruca,et al.  The Effect of Electronic Markets on Forecasts of New Product Success , 2003, Inf. Syst. Frontiers.

[47]  Christof Weinhardt,et al.  Short-selling in prediction markets , 2012 .

[48]  Martin Spann,et al.  Sports forecasting: a comparison of the forecast accuracy of prediction markets, betting odds and tipsters , 2009 .

[49]  Ronald Peeters,et al.  Incentives in Prediction Markets , 2012 .

[50]  Kay-Yut Chen,et al.  New Product Blockbusters: The Magic and Science of Prediction Markets , 2007 .

[51]  Thomas A. Rietz,et al.  Prediction market accuracy in the long run , 2008 .

[52]  Steven D. Levitt,et al.  Winning Isn't Everything: Corruption in Sumo Wrestling , 2002 .

[53]  J. Wolfers,et al.  How Prediction Markets Can Save Event Studies , 2008, SSRN Electronic Journal.

[54]  David Paton,et al.  Market Efficiency in Person-to-Person Betting , 2006 .

[55]  Stefan Luckner,et al.  Prediction Markets: Fundamentals, Key Design Elements, and Applications , 2008, Bled eConference.

[56]  David M. Pennock,et al.  Prediction Markets: Does Money Matter? , 2004, Electron. Mark..

[57]  Ramakrishnan Ramanathan,et al.  Multicriteria Methods for Technology Foresight , 2003 .

[58]  Robert Forsythe,et al.  Anatomy of an Experimental Political Stock Market , 1992 .

[59]  Ahti Salo,et al.  A group support system for foresight processes , 2004 .

[60]  Stefan Luckner,et al.  Prediction markets for foresight , 2010 .

[61]  A. Raftery,et al.  Strictly Proper Scoring Rules, Prediction, and Estimation , 2007 .

[62]  Stefan Luckner,et al.  On the Forecast Accuracy of Sports Prediction Markets , 2006, Negotiation, Auctions, and Market Engineering.

[63]  J. L. Salmeron,et al.  Foresighting key areas in the Information Technology industry , 2008 .

[64]  Lian Jian,et al.  Aggregation and Manipulation in Prediction Markets: Effects of Trading Mechanism and Information Distribution , 2012, Manag. Sci..

[65]  Andrzej M. J. Skulimowski Discovering Complex System Dynamics with Intelligent Data Retrieval Tools , 2011, IScIDE.

[66]  Koleman Strumpf,et al.  Historical presidential betting markets , 2004 .

[67]  Paul C. Tetlock,et al.  The Promise of Prediction Markets , 2008, Science.

[68]  Gene Rowe,et al.  Judgment change during Delphi-like procedures: The role of majority influence, expertise, and confidence , 2004 .

[69]  B. Skiera,et al.  Sourcing, Filtering, and Evaluating New Product Ideas: An Empirical Exploration of the Performance of Idea Markets , 2012 .

[70]  Peter Bossaerts,et al.  Expectations and learning in Iowa , 2000 .

[71]  F. Nelson,et al.  Use of prediction markets to forecast infectious disease activity. , 2007, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[72]  F. Hayek The economic nature of the firm: The use of knowledge in society , 1945 .

[73]  Martin Spann,et al.  Internet-Based Virtual Stock Markets for Business Forecasting , 2003, Manag. Sci..

[74]  William D. Nordhaus,et al.  Forecasting Efficiency: Concepts and Applications , 1987 .

[75]  Andreas Graefe,et al.  Prediction Markets and the “Trough of Disillusionment” , 2011 .

[76]  George Wright,et al.  Delphi: A reevaluation of research and theory , 1991 .

[77]  Sebastian Diemer,et al.  Real-money vs. play-money forecasting accuracy in online prediction markets - empirical insights from Ipredict , 2010 .

[78]  M. Flannery,et al.  Macroeconomic Factors Do Influence Aggregate Stock Returns , 2002 .

[79]  David M. Pennock,et al.  Information markets vs. opinion pools: an empirical comparison , 2005, EC '05.

[80]  J. D. Christiansen,et al.  Prediction Markets: Practical Experiments in Small Markets and Behaviours Observed , 2012 .

[81]  Leonard Lee,et al.  Feeling the Future: The Emotional Oracle Effect , 2011 .

[82]  E. Elton,et al.  Economic News and Bond Prices: Evidence from the U.S. Treasury Market , 2001, Journal of Financial and Quantitative Analysis.

[83]  Christian Slamka,et al.  Prediction Market Performance and Market Liquidity: A Comparison of Automated Market Makers , 2013, IEEE Transactions on Engineering Management.

[84]  Thomas Dohmen The Influence of Social Forces: Evidence from the Behavior of Football Referees , 2008 .

[85]  Justin Wolfers,et al.  Using Markets to Inform Policy: The Case of the Iraq War , 2009 .

[86]  Christer Carlsson,et al.  Industry Foresight with Intelligent Agents , 1999 .

[87]  Robert Forsythe,et al.  Using Market Prices to Predict Election Results: The 1993 UBC Election Stock Market , 1995 .

[88]  Kajal Lahiri,et al.  How quickly do forecasters incorporate news? Evidence from cross-country surveys , 2006 .

[89]  Andrew C. Pollock,et al.  An application of probability judgement accuracy measures to currency forecasting , 1996 .

[90]  Andreas Graefe,et al.  Prediction Markets for Forecasting Drug Development , 2010 .

[91]  Wolfgang Jank,et al.  Second-Generation Prediction Markets for Information Aggregation: A Comparison of Payoff Mechanisms , 2009 .

[92]  David Rothschild,et al.  Forecasting Elections Comparing Prediction Markets, Polls, and Their Biases , 2009 .

[93]  S. Frederick,et al.  Intuitive Biases in Choice versus Estimation: Implications for the Wisdom of Crowds , 2011 .

[94]  Gordon Leitch,et al.  Economic Forecast Evaluation: Profits versus the Conventional Error Measures , 1991 .

[95]  Janine Hansen,et al.  Manipulation in political stock markets – preconditions and evidence , 2004 .

[96]  J. Scott Armstrong,et al.  University of Pennsylvania Scholarlycommons Comparing Face-to-face Meetings, Nominal Groups, Delphi and Prediction Markets on an Estimation Task Comparing Face-to-face Meetings, Nominal Groups, Delphi and Prediction Markets on an Estimation Task Comparing Face-to-face Meetings, Nominal Groups, Delph , 2022 .

[97]  H. Ernst,et al.  Identification of Lead Users for Consumer Products via Virtual Stock Markets , 2009 .

[98]  Frank T. Magiera,et al.  Macroeconomic Factors Do Influence Aggregate Stock Returns , 2002 .

[99]  M. Keith Chen,et al.  Modeling a Presidential Prediction Market , 2008, Manag. Sci..

[100]  Murray Turoff,et al.  Delphi: A brief look backward and forward , 2011 .

[101]  Richard Donkin,et al.  The End of Management , 2010 .

[102]  Michael P. Clements Evaluating the Rationality of Fixed-Event Forecasts , 1997 .

[103]  Jacob Goldenberg,et al.  Inevitably reborn: The reawakening of extinct innovations ☆ , 2004 .

[104]  Dilek Önkal,et al.  Professional vs Amateur Judgment Accuracy: The Case of Foreign Exchange Rates , 2003 .

[105]  Kenneth Oliven,et al.  Suckers Are Born but Markets Are Made: Individual Rationality, Arbitrage, and Market Efficiency on an Electronic Futures Market , 2004, Manag. Sci..

[106]  Wolfgang Jank,et al.  Second-Generation Prediction Markets for Information Aggregation: A Comparison of Payoff Mechanisms: Second-Generation Prediction Markets , 2012 .

[107]  David M. Pennock,et al.  The Tech Buzz Game , 2005, Computer.

[108]  Andrzej M. J. Skulimowski,et al.  A Foresight Support System to Manage Knowledge on Information Society Evolution , 2012, SocInfo.

[109]  Andrew B. Whinston,et al.  Supply chain information sharing in a macro prediction market , 2006, Decis. Support Syst..

[110]  Christophe Boone,et al.  Top Management Team Diversity and Firm Performance: Moderators of Functional-Background and Locus-of-Control Diversity , 2009, Manag. Sci..

[111]  J. Scott Armstrong,et al.  Methods to Elicit Forecasts from Groups: Delphi and Prediction Markets Compared , 2007 .

[112]  Dilek Önkal,et al.  Scenarios as channels of forecast advice , 2013 .

[113]  William I. Notz,et al.  Statistical Tests of Real-Money versus Play-Money Prediction Markets , 2006, Electron. Mark..

[114]  Justin Wolfers,et al.  Macroeconomic Derivatives: An Initial Analysis of Market-Based Macro Forecasts, Uncertainty and Risk , 2005, SSRN Electronic Journal.

[115]  Gerhard Ortner,et al.  Political Stock Markets: The Austrian Experiences , 1995 .

[116]  Michael E. Sobel,et al.  Forecasting the Vote: A Theoretical Comparison of Election Markets and Public Opinion Polls , 2004, Political Analysis.

[117]  Tomaso Poggio,et al.  Securities Trading of Concepts (STOC) , 2011 .