Classification trees and decision-analytic feedforward control: a case study from the video game industry

The objective of this paper is to use a challenging real-world problem to illustrate how a probabilistic predictive model can provide the foundation for decision-analytic feedforward control. Commercial data mining software and sales data from a market research firm are used to create a predictive model of market success in the video game industry. A procedure is then described for transforming the classification trees into a decision-analytic model that can be solved to produce a value-maximizing game development policy. The video game example shows how the compact predictive models created by data mining algorithms can help to make decision-analytic feedforward control feasible, even for large, complex problems. However, the example also highlights the bounds placed on the practicality of the approach due to combinatorial explosions in the number of contingencies that have to be modeled. We show, for example, how the “option value” of sequels creates complexity that is effectively impossible to address using conventional decision analysis tools.

[1]  Ralph L. Keeney,et al.  Decisions with multiple objectives: preferences and value tradeoffs , 1976 .

[2]  Dorian Pyle,et al.  Data Preparation for Data Mining , 1999 .

[3]  Herbert A. Simon,et al.  The new science of management decision , 1960 .

[4]  R. L. Keeney,et al.  Decisions with Multiple Objectives: Preferences and Value Trade-Offs , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[5]  L. Trigeorgis Real Options: Managerial Flexibility and Strategy in Resource Allocation , 1996 .

[6]  Robert T. Clemen,et al.  Making Hard Decisions: An Introduction to Decision Analysis , 1997 .

[7]  Kun Zhang,et al.  Discovering Unrevealed Properties of Probability Estimation Trees: On Algorithm Selection and Performance Explanation , 2006, Sixth International Conference on Data Mining (ICDM'06).

[8]  T. Wilson,et al.  Real-timestrategy:Evolutionarygamedevelopment , 2006 .

[9]  Christen Rose-Anderssen,et al.  New Product Development as a Complex Adaptive System of Decisions , 2006 .

[10]  Heikki Topi,et al.  A Review of Software Packages for Data Mining , 2003 .

[11]  Shimon Schocken,et al.  Reframing Decision Problems: A Graph-Grammar Approach , 1991, Inf. Syst. Res..

[12]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[13]  Howard Raiffa,et al.  Decision analysis: introductory lectures on choices under uncertainty. 1968. , 1969, M.D.Computing.

[14]  Padhraic Smyth,et al.  Business applications of data mining , 2002, CACM.

[15]  P. Tufano,et al.  A real-world way to manage real options. , 2004, Harvard business review.

[16]  Chris F. Kemerer,et al.  Environmental Volatility, Development Decisions, and Software Volatility: A Longitudinal Analysis , 2006, Manag. Sci..

[17]  Charles A. Holloway Decision Making Under Uncertainty: Models and Choices , 1979 .

[18]  Michael Brydon,et al.  Evaluating strategic options using decision-theoretic planning , 2006, Inf. Technol. Manag..

[19]  Pedro M. Domingos,et al.  Tree Induction for Probability-Based Ranking , 2003, Machine Learning.

[20]  Naoki Abe,et al.  Sequential cost-sensitive decision making with reinforcement learning , 2002, KDD.

[21]  Diane M. Lander,et al.  Challenges to the practical implementation of modeling and valuing real options , 1998 .

[22]  E. Sadler‐Smith,et al.  The Role of Intuition in Strategic Decision Making , 2009 .

[23]  Bianca Zadrozny,et al.  Obtaining calibrated probability estimates from decision trees and naive Bayesian classifiers , 2001, ICML.

[24]  Rodney A. Brooks,et al.  Intelligence Without Reason , 1991, IJCAI.

[25]  T. Davenport,et al.  Data to Knowledge to Results: Building an Analytic Capability , 2001 .

[26]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[27]  James E. Smith,et al.  Options in the Real World: Lessons Learned in Evaluating Oil and Gas Investments , 1999, Oper. Res..

[28]  T. Luehrman,et al.  Strategy as a portfolio of real options. , 1998, Harvard business review.

[29]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[30]  Maytal Saar-Tsechansky,et al.  Economical active feature-value acquisition through Expected Utility estimation , 2005, UBDM '05.