The Effectiveness of Simple Decision Heuristics: Forecasting Commercial Success for Early-Stage Ventures

We investigate the decision heuristics used by experts to forecast that early-stage ventures are subsequently commercialized. Experts evaluate 37 project characteristics and subjectively combine data on all cues by examining both critical flaws and positive factors to arrive at a forecast. A conjunctive model is used to describe their process, which sums "good" and "bad" cue counts separately. This model achieves a 91.8% forecasting accuracy of the experts' correct forecasts. The model correctly predicts 86.0% of outcomes in out-of-sample, out-of-time tests. Results indicate that reasonably simple decision heuristics can perform well in a natural and very difficult decision-making context.

[1]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[2]  R. Skov,et al.  Information-gathering processes: Diagnosticity, hypothesis-confirmatory strategies, and perceived hypothesis confirmation. , 1986 .

[3]  J. Alba,et al.  The Effects of Frequency Knowledge On Consumer Decision Making , 1987 .

[4]  John W. Payne,et al.  The adaptive decision maker: Name index , 1993 .

[5]  R. Hogarth,et al.  Confidence in judgment: Persistence of the illusion of validity. , 1978 .

[6]  Gershon Ben-Shakhar,et al.  Seek and Ye Shall Find: Test Results Are What You Hypothesize They Are , 1998 .

[7]  J. Shanteau.  Psychological characteristics and strategies of expert decision makers , 1988 .

[8]  T. Åstebro,et al.  The Return to Independent Invention: Evidence of Unrealistic Optimism, Risk Seeking or Skewness Loving? , 2003 .

[9]  D. Goldstein,et al.  How good are simple heuristics , 1999 .

[10]  Eric J. Johnson,et al.  The adaptive decision maker , 1993 .

[11]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[12]  J. Angrist,et al.  Digitized by the Internet Archive in 2011 with Funding from Estimation of Limited-dependent Variable Models with Dummy Endogenous Regressors: Simple Strategies for Empirical Practice , 2011 .

[13]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[14]  H. Raiffa,et al.  Decisions with Multiple Objectives , 1993 .

[15]  Thomas B. Astebro,et al.  Statistical Decision-Making Models and Treatment Effects , 2004 .

[16]  A. H. Murphy,et al.  Probability Forecasting in Meteorology , 1984 .

[17]  Yigal Gerchak,et al.  CALCULATING THE EXPECTATION AND VARIANCE OF THE PRESENT VALUE FOR A RANDOM PROFIT STREAM OF UNCERTAIN DURATION , 2000 .

[18]  Spyros Makridakis,et al.  The M3-Competition: results, conclusions and implications , 2000 .

[19]  B. Fischhoff,et al.  Hindsight is not equal to foresight: The effect of outcome knowledge on judgment under uncertainty. , 1975 .

[20]  Thomas B. Astebro,et al.  Calibration accuracy of a judgmental process that predicts the commercial success of new product ideas , 2007 .

[21]  Thomas B. Astebro,et al.  The Social Rate of Return to Canada's Inventor's Assistance Program , 1999 .

[22]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[23]  G. Meyer,et al.  The potential of actuarial decision models: Can they improve the venture capital investment decision? , 2000 .

[24]  L. R. Goldberg.  Simple models or simple processes? Some research on clinical judgments. , 1968, The American psychologist.

[25]  A. Tversky,et al.  Extensional versus intuitive reasoning: the conjunction fallacy in probability judgment , 1983 .

[26]  M. Sobel,et al.  Identification Problems in the Social Sciences. , 1996 .

[27]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[28]  James Shanteau,et al.  Expert Judgment: Is More Information Better? , 1987 .

[29]  Yigal Gerchak,et al.  Profitable Advice: the Value of Information Provided by Canada's Inventor's Assistance Program , 2001 .

[30]  B. Fischhoff,et al.  I knew it would happen: Remembered probabilities of once—future things , 1975 .

[31]  W. Thorngate.  Efficient decision heuristics. , 1980 .

[32]  Peter M. Todd,et al.  Categorization by elimination : a fast and frugal approach tocategorization , 1997 .

[33]  Gerald G. Udell.  Invention Evaluation Services: A Review of the State of the Art , 1989 .

[34]  R. Dawes.  Judgment under uncertainty: The robust beauty of improper linear models in decision making , 1979 .

[35]  R. Dawes,et al.  Heuristics and Biases: Clinical versus Actuarial Judgment , 2002 .

[36]  Kenneth G. Baker,et al.  Modeling New Product Screening Decisions , 1986 .

[37]  L. J. Chapman,et al.  Genesis of popular but erroneous psychodiagnostic observations. , 1967, Journal of abnormal psychology.

[38]  A. Tversky.  Elimination by aspects: A theory of choice. , 1972 .

[39]  L. J. Chapman,et al.  Judgment under uncertainty: Test results are what you think they are , 1982 .

[40]  Colin F. Camerer,et al.  General conditions for the success of bootstrapping models , 1981 .

[41]  A. Tversky,et al.  Prospect Theory : An Analysis of Decision under Risk Author ( s ) : , 2007 .

[42]  Peter M. Todd,et al.  Simple Inference Heuristics versus Complex Decision Machines , 1999, Minds and Machines.

[43]  Thomas B. Astebro.  Basic Statistics on the Success Rate and Profits for Independent Inventors , 1998 .

[44]  B. Dosher,et al.  Strategies for multiattribute binary choice. , 1983, Journal of experimental psychology. Learning, memory, and cognition.

[45]  P. Meehl,et al.  Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. , 1996 .

[46]  B. Fischhoff,et al.  Judgment under uncertainty: Debiasing , 1982 .

[47]  Sholom M. Weiss,et al.  Maximizing the Predictive Value of Production Rules , 1990, Artif. Intell..

[48]  D. Faust.  The limits of scientific reasoning , 1984 .

[49]  P. Hoffman.  The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[50]  K. R. Hammond,et al.  COGNITIVE DEPENDENCE ON LINEAR AND NONLINEAR CUES. , 1965, Psychological review.

[51]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[52]  Hillel J. Einhorn,et al.  Expert measurement and mechanical combination , 1972 .

[53]  Thomas Åstebro,et al.  Key success factors for technological entrepreneurs' R&D projects , 2004, IEEE Transactions on Engineering Management.

[54]  J. Shanteau.  How much information does an expert use? Is it relevant? , 1992 .