Can experts really assess future technology success? A neural network and Bayesian analysis of early stage technology proposals

This paper compares experts' assessments to a set of structural variables to determine whether each effectively predicts technology success. Using 69 homeland security and defense-related technologies, expert reviewers scored each technology on various dimensions as part of a government grant funding process. These technologies were tracked over 3 years and degrees of success recorded. Different predictive models were estimated using an artificial neural network technique, the Bayesian Data Reduction Algorithm, and two regression equations. The results suggest that experts provide little predictive power, and that a reasonable technology success model can be estimated using a limited set of structural variables.

[1]  Emmett W. Eldred,et al.  Chapter 8 – Technology Management , 1996 .

[2]  Abbie Griffin,et al.  Modeling and measuring product development cycle time across industries , 1997 .

[3]  Stephen W. Looney,et al.  A statistical technique for comparing the accuracies of several classifiers , 1988, Pattern Recognit. Lett..

[4]  Lutz Prechelt,et al.  A quantitative study of experimental evaluations of neural network learning algorithms: Current research practice , 1996, Neural Networks.

[5]  José Manuel Benítez,et al.  Interpretation of artificial neural networks by means of fuzzy rules , 2002, IEEE Trans. Neural Networks.

[6]  Dean A. Shepherd,et al.  The nature of information and overconfidence on venture capitalists' decision making , 2001 .

[7]  Emmett W. Eldred,et al.  Commercializing New Technology—I , 1997 .

[8]  Joel A. C. Baum,et al.  Picking winners or building them? Alliance, intellectual, and human capital as selection criteria in venture financing and performance of biotechnology startups , 2004 .

[9]  John Preston,et al.  Winning at New Products , 1988 .

[10]  Robert H. Keeley,et al.  Predictors of success in new technology based ventures , 1990 .

[11]  Robert G. Cooper,et al.  Optimizing the State-Gate Process: What Best-Practice Companies Do-II: Leading Companies Are Creating More Rigorous Go/kill Decision Points and Implementing More Effective Portfolio Management , 2002 .

[12]  G. Meyer,et al.  The potential of actuarial decision models: Can they improve the venture capital investment decision? , 2000 .

[13]  Jay J.J. Christensen-Szalanski,et al.  The hindsight bias: A meta-analysis , 1991 .

[14]  John H. Friar,et al.  Factors for success in R&D projects and new product innovation: a contextual framework , 1997 .

[15]  Craig S. Galbraith,et al.  Performance analysis of the Bayesian data reduction algorithm , 2009, Int. J. Data Min. Model. Manag..

[16]  Peter A. Koen,et al.  Technology Stage Gate: A Structured Process for Managing High Risk, New Technology Projects , 2001 .

[17]  Arthur Flexer,et al.  Statistical evaluation of neural networks experiments: Minimum requirements and current practice , 1994 .

[18]  Alfredo Vellido,et al.  Neural networks in business: a survey of applications (1992–1998) , 1999 .

[19]  Fiorenzo Franceschini,et al.  Quality and innovation: A conceptual model of their interaction , 1995 .

[20]  Muammer Ozer,et al.  A survey of new product evaluation models , 1999 .

[21]  Abbie Griffin,et al.  The PDMA toolbook for new product development , 2002 .

[22]  Robert G. Cooper,et al.  Winning at new products : accelerating the process from idea to launch , 2001 .

[23]  M. R. Chernick,et al.  Application of bootstrap and other resampling techniques: Evaluation of classifier performance , 1985, Pattern Recognit. Lett..

[24]  Michael Y. Hu,et al.  Estimating breast cancer risks using neural networks , 2002, J. Oper. Res. Soc..

[25]  Louise A. Heslop,et al.  Development of a Technology Readiness Assessment Measure: The Cloverleaf Model of Technology Transfer , 2001 .

[26]  Norma Harrison,et al.  Technology management , 2001 .

[27]  B. Zirger,et al.  A study of success and failure in product innovation: The case of the U.S. electronics industry , 1984, IEEE Transactions on Engineering Management.

[28]  Ramazan Gençay,et al.  Software review: MATLAB neural network toolbox , 2001 .

[29]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[30]  F. Timothy Janis,et al.  Moving R&D to the Marketplace, A Guidebook for Technology Transfer Managers , 1993 .

[31]  Peter Willett,et al.  Bayesian classification and feature reduction using uniform Dirichlet priors , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[32]  Holger R. Maier,et al.  Neural networks for the prediction and forecasting of water resource variables: a review of modelling issues and applications , 2000, Environ. Model. Softw..

[33]  Robert G. Cooper,et al.  Product Leadership: Creating And Launching Superior New Products , 1998 .

[34]  P. Davidsson,et al.  The role of social and human capital among nascent entrepreneurs , 2000 .

[35]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Robert G. Cooper,et al.  The Dimensions of Industrial New Product Success and Failure , 1979 .

[37]  Fred Collopy,et al.  How effective are neural networks at forecasting and prediction? A review and evaluation , 1998 .

[38]  Robert G. Cooper,et al.  Optimizing the Stage-Gate Process: What Best-Practice Companies Do—I , 2002 .

[39]  Paolo Crippa,et al.  Artificial Neural Networks as Approximators of Stochastic Processes , 1998 .

[40]  Thomas Åstebro,et al.  Key success factors for technological entrepreneurs' R&D projects , 2004, IEEE Transactions on Engineering Management.

[41]  J. Linton,et al.  Analysis, ranking and selection of R&D projects in a portfolio , 2002 .

[42]  Peter Willett,et al.  Performance considerations for a combined information classification test using Dirichlet priors , 1999, IEEE Trans. Signal Process..

[43]  Scott Shane,et al.  Special Issue on University Entrepreneurship and Technology Transfer: Organizational Endowments and the Performance of University Start-ups , 2002, Manag. Sci..

[44]  P. Koen,et al.  1 Fuzzy Front End : Effective Methods , Tools , and Techniques , 2002 .

[45]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[46]  Dean A. Shepherd,et al.  Vcs' Decision Processes: Evidence Suggesting More Experience May Not Always Be Better , 2003 .

[47]  Theodore F. Schoenborn,et al.  So you want to set up a technology evaluation program? , 2000 .

[48]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[49]  R. Calantone,et al.  Determinants of New Product Performance: A Review and Meta-Analysis , 1994 .

[50]  Christopher R. Blake,et al.  Survivorship Bias and Mutual Fund Performance , 1995 .