Feature selection using Bayesian and multiclass Support Vector Machines approaches: Application to bank risk prediction

This paper presents methods of banks discrimination according to the rate of NonPerforming Loans (NPLs), using Gaussian Bayes models and different approaches of multiclass Support Vector Machines (SVM). This classification problem involves many irrelevant variables and comparatively few training instances. New variable selection strategies are proposed. They are based on Gaussian marginal densities for Bayesian models and ranking scores derived from multiclass SVM. The results on both toy data and real-life problem of banks classification demonstrate a significant improvement of prediction performance using only a few variables. Moreover, Support Vector Machines approaches are shown to be superior to Gaussian Bayes models.

[1]  Bhekisipho Twala,et al.  Multiple classifier application to credit risk assessment , 2010, Expert Syst. Appl..

[2]  Hiok Chai Quek,et al.  FCMAC-EWS: A bank failure early warning system based on a novel localized pattern learning and semantically associative fuzzy neural network , 2008, Expert Syst. Appl..

[3]  Vadlamani Ravi,et al.  Threshold accepting trained principal component neural network and feature subset selection: Application to bankruptcy prediction in banks , 2008, Appl. Soft Comput..

[4]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[5]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[6]  Sumit Sarkar,et al.  Bayesian Models for Early Warning of Bank Failures , 2001, Manag. Sci..

[7]  Boonserm Kijsirikul,et al.  Reordering adaptive directed acyclic graphs: an improved algorithm for multiclass support vector machines , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[8]  Christophe J. Godlewski.  Regulatory and Institutional Determinants of Credit Risk Taking and a Bank's Default in Emerging Market Economies , 2006 .

[9]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[10]  Wei Ge,et al.  Effects of feature construction on classification performance: An empirical study in bank failure prediction , 2009, Expert Syst. Appl..

[11]  Anis Ben Ishak,et al.  Sélection de variables par les machines à vecteurs supports pour la discrimination binaire et multiclasse en grande dimension , 2007 .

[12]  J. Sinkey.  A MULTIVARIATE STATISTICAL ANALYSIS OF THE CHARACTERISTICS OF PROBLEM BANKS , 1975 .

[13]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[14]  Lawrence M. Seiford,et al.  Forecasting Bank Failure: A Non-Parametric Frontier Estimation Approach , 1994, Recherches économiques de Louvain.

[15]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[16]  Vadlamani Ravi,et al.  Differential evolution trained wavelet neural networks: Application to bankruptcy prediction in banks , 2009, Expert Syst. Appl..

[17]  G. Benston,et al.  How should banks account for loan losses , 2005 .

[18]  Alain Rakotomamonjy,et al.  Variable Selection Using SVM-based Criteria , 2003, J. Mach. Learn. Res..

[19]  Serpil Canbas,et al.  Prediction of commercial bank failure via multivariate statistical analysis of financial structures: The Turkish case , 2005, Eur. J. Oper. Res..

[20]  H. Altay Güvenir,et al.  Voting features based classifier with feature construction and its application to predicting financial distress , 2010, Expert Syst. Appl..

[21]  Timothy B. Bell,et al.  Neural nets or the logit model? A comparison of each model’s ability to predict commercial bank failures , 1997 .

[22]  Rebel A. Cole,et al.  Separating the likelihood and timing of bank failure , 1995 .

[23]  E. Altman.  Predicting performance in the savings and loan association industry , 1977 .

[24]  Melody Y. Kiang,et al.  Managerial Applications of Neural Networks: The Case of Bank Failure Predictions , 1992 .

[25]  B. Ghattas,et al.  Sélection de variables pour la classification binaire en grande dimension: comparaisons et application aux domées de biopuces , 2008 .

[26]  Jeffrey A. Clark,et al.  Off-site monitoring systems for predicting bank underperformance: a comparison of neural networks, discriminant analysis, and professional human judgment , 2001, Intell. Syst. Account. Finance Manag..

[27]  Daniel Martin,et al.  Early warning of bank failure: A logit regression approach , 1977 .

[28]  Francisco González,et al.  Cross-country determinants of bank income smoothing by managing loan loss provisions , 2008 .

[29]  James W. Kolari,et al.  Predicting large US commercial bank failures , 2002 .

[30]  Wanchai Rivepiboon,et al.  Reordering Adaptive Directed Acyclic Graphs for Multiclass Support Vector Machines , 2003, J. Adv. Comput. Intell. Intell. Informatics.

[31]  R. C. West.  A factor-analytic approach to bank condition , 1985 .

[32]  W. R. Lane,et al.  An application of the cox proportional hazards model to bank failure , 1986 .

[33]  Ömer Kaan Baykan,et al.  Predicting bank financial failures using neural networks, support vector machines and multivariate statistical methods: A comparative analysis in the sample of savings deposit insurance fund (SDIF) transferred banks in Turkey , 2009, Expert Syst. Appl..

[34]  Ulrich H.-G. Kreßel,et al.  Pairwise classification and support vector machines , 1999 .

[35]  Anthony M. Santomero,et al.  Risk in Banking and Capital Regulation , 1988 .

[36]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[37]  Vadlamani Ravi,et al.  Bankruptcy prediction in banks and firms via statistical and intelligent techniques - A review , 2007, Eur. J. Oper. Res..