Neural network ensemble strategies for financial decision applications

Considerable research effort has been expended to identify more accurate models for decision support systems in financial decision domains including credit scoring and bankruptcy prediction. The focus of this earlier work has been to identify the "single best" prediction model from a collection that includes simple parametric models, nonparametric models that directly estimate data densities, and nonlinear pattern recognition models such as neural networks. Recent theories suggest this work may be misguided in that ensembles of predictors provide more accurate generalization than the reliance on a single model. This paper investigates three recent ensemble strategies: crossvalidation, bagging, and boosting. We employ the multilayer perceptron neural network as a base classifier. The generalization ability of the neural network ensemble is found to be superior to the single best model for three real world financial decision applications.

[1]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[2]  J. Ross Quinlan,et al.  Simplifying decision trees , 1987, Int. J. Hum. Comput. Stud..

[3]  Jie Zhang,et al.  Inferential estimation of polymer quality using bootstrap aggregated neural networks , 1999, Neural Networks.

[4]  David West,et al.  Neural network credit scoring models , 2000, Comput. Oper. Res..

[5]  Jonathan Crook,et al.  Credit Scoring Models in the Credit Union Environment Using Neural Networks and Genetic Algorithms , 1997 .

[6]  So Young Sohn,et al.  Data fusion, ensemble and clustering to improve the classification accuracy for the severity of road traffic accidents in Korea , 2003 .

[7]  Loretta J. Mester.  What's the point of credit scoring? , 1997 .

[8]  Yoshua Bengio,et al.  Boosting Neural Networks , 2000, Neural Computation.

[9]  B. Efron,et al.  A Leisurely Look at the Bootstrap, the Jackknife, and , 1983 .

[10]  Pádraig Cunningham,et al.  Stability problems with artificial neural networks and the ensemble solution , 2000, Artif. Intell. Medicine.

[11]  Ray L. Somorjai,et al.  Application of Several Methods of Classification Fusion to Magnetic Resonance Spectra , 1996, Connect. Sci..

[12]  R. Centor.  Signal Detectability , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[13]  TamKar Yan,et al.  Managerial Applications of Neural Networks , 1992 .

[14]  Pamela K. Coats,et al.  A neural network for classifying the financial health of a firm , 1995 .

[15]  R. Schapire.  The Strength of Weak Learnability , 1990, Machine Learning.

[16]  Alan K. Reichert,et al.  An Examination of the Conceptual Issues Involved in Developing Credit-Scoring Models , 1983 .

[17]  J. Ross Quinlan,et al.  Simplifying Decision Trees , 1987, Int. J. Man Mach. Stud..

[18]  G DietterichThomas.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees , 2000 .

[19]  Lars Kai Hansen,et al.  Neural Network Ensembles , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Daniel E. O'Leary,et al.  Knowledge Acquisition From Multiple Experts: An Empirical Study , 1998 .

[21]  Yu-Bin Yang,et al.  Lung cancer cell identification based on artificial neural network ensembles , 2002, Artif. Intell. Medicine.

[22]  Yoichi Hayashi,et al.  Combining neural network predictions for medical diagnosis , 2002, Comput. Biol. Medicine.

[23]  Melody Y. Kiang,et al.  Managerial Applications of Neural Networks: The Case of Bank Failure Predictions , 1992 .

[24]  H. Frydman,et al.  Introducing Recursive Partitioning for Financial Classification: The Case of Financial Distress , 1985 .

[25]  E. Mine Cinar,et al.  Neural Networks: A New Tool for Predicting Thrift Failures , 1992 .

[26]  Michael Y. Hu,et al.  Explaining consumer choice through neural networks: The stacked generalization approach , 2003, Eur. J. Oper. Res..

[27]  Thomas L. Ainscough,et al.  Neural networks in business. , 1997 .

[28]  Robert E. Schapire,et al.  The strength of weak learnability , 1990, Mach. Learn..

[29]  Parag C. Pendharkar,et al.  An empirical study of design and testing of hybrid evolutionary–neural approach for classification , 2001 .

[30]  Clive L. Dym,et al.  Knowledge Acquisition from Multiple Experts , 1984, AI Mag..

[31]  Edward I. Altman,et al.  Corporate distress diagnosis: Comparisons using linear discriminant analysis and neural networks (the Italian experience) , 1994 .

[32]  D. Hand,et al.  A k-nearest-neighbour classifier for assessing consumer credit risk , 1996 .

[33]  Nathan Intrator,et al.  Boosting Regression Estimators , 1999, Neural Computation.

[34]  William Edward Henley,et al.  Statistical aspects of credit scoring , 1995 .

[35]  David West,et al.  Credit Scoring Using Supervised and Unsupervised Neural Networks , 2002 .

[36]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[37]  Daniel P. Lopresti,et al.  Improving classifier performance through repeated sampling , 1997, Pattern Recognit..

[38]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[39]  Selwyn Piramuthu,et al.  Financial credit-risk evaluation with neural and neurofuzzy systems , 1999, Eur. J. Oper. Res..

[40]  Herbert L. Jensen.  Using Neural Networks for Credit Scoring , 1992 .

[41]  Leo Breiman,et al.  Prediction Games and Arcing Algorithms , 1999, Neural Computation.

[42]  R. Tibshirani,et al.  Combining Estimates in Regression and Classification , 1996 .

[43]  Jie Zhang,et al.  Developing robust non-linear models through bootstrap aggregated neural networks , 1999, Neurocomputing.

[44]  Alexander Gammerman,et al.  Machine-learning algorithms for credit-card applications , 1992 .

[45]  Vijay S. Desai,et al.  A comparison of neural networks and linear scoring models in the credit union environment , 1996 .

[46]  Eric Rosenberg,et al.  Quantitative Methods in Credit Management: A Survey , 1994, Oper. Res..

[47]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[48]  J. Gupta,et al.  Neural Networks in Business: Techniques and Applications , 2001 .

[49]  Jatinder N. D. Gupta,et al.  Neural networks in business: techniques and applications for the operations researcher , 2000, Comput. Oper. Res..