Are we modelling the right thing? The impact of incorrect problem specification in credit scoring

Classification and regression models are widely used by mainstream credit granting institutions to assess the risk of customer default. In practice, the objectives used to derive model parameters and the business objectives used to assess models differ. Models parameters are determined by minimising some function or error or by maximising likelihood, but performance is assessed using global measures such as the GINI coefficient, or the misclassification rate at a specific point in the score distribution. This paper seeks to determine the impact on performance that results from having different objectives for model construction and model assessment. To do this a genetic algorithm (GA) is utilized to generate linear scoring models that directly optimise business measures of interest. The performance of the GA models is then compared to those constructed using logistic and linear regression. Empirical results show that all models perform similarly well, suggesting that modelling and business objectives are well aligned.

[1]  Steven Finlay,et al.  The Management of Consumer Credit: Theory and Practice , 2008 .

[2]  Colin Reeves.  Genetic Algorithms , 2003, Handbook of Metaheuristics.

[3]  Colin R. Reeves,et al.  A genetic algorithm for flowshop sequencing , 1995, Comput. Oper. Res..

[4]  J. Crook,et al.  Credit scoring using neural and evolutionary techniques , 2000 .

[5]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[6]  David Levine.  Genetic Algorithms: A Practitioner's View , 1997, INFORMS J. Comput..

[7]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[8]  R. H. Storer,et al.  Developing Fitter Genetic Algorithms , 1997 .

[9]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[10]  Ashish Tiwari,et al.  A greedy genetic algorithm for the quadratic assignment problem , 2000, Comput. Oper. Res..

[11]  Terence C. Fogarty,et al.  Evolving Bayesian classifiers for credit control—a comparison with other machine-learning methods , 1993 .

[12]  C. Reeves.  Modern heuristic techniques for combinatorial problems , 1993 .

[13]  Ravindra K. Ahuja,et al.  Developing Fitter Genetic Algorithms , 1997, INFORMS J. Comput..

[14]  Alden H. Wright,et al.  Genetic Algorithms for Real Parameter Optimization , 1990, FOGA.

[15]  Shih-Wei Lin,et al.  A hybrid approach for single-machine tardiness problems with sequence-dependent setup times , 2008, J. Oper. Res. Soc..

[16]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[17]  G. D. Smith,et al.  Solving the Graphical Steiner Tree Problem Using Genetic Algorithms , 1993 .

[18]  Jonathan Crook,et al.  Credit Scoring Models in the Credit Union Environment Using Neural Networks and Genetic Algorithms , 1997 .

[19]  Niall M. Adams,et al.  Defining attributes for scorecard construction in credit scoring , 2000 .

[20]  Jd Landes,et al.  Recent Developments in J Ic Testing , 1977 .

[21]  David Coley,et al.  Introduction to Genetic Algorithms for Scientists and Engineers , 1999 .

[22]  Kenneth A. De Jong,et al.  An Analysis of Multi-Point Crossover , 1990, FOGA.

[23]  L C Thomas,et al.  Recalibrating scorecards , 2001, J. Oper. Res. Soc..

[24]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[25]  John Fox,et al.  Nonparametric simple regression , 2000 .

[26]  P. Lovie,et al.  The flat maximum effect and linear scoring models for prediction , 1986 .

[27]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[28]  David J. Hand,et al.  Statistical Classification Methods in Consumer Credit Scoring: a Review , 1997 .

[29]  Colin R. Reeves,et al.  Genetic Algorithms: Principles and Perspectives: A Guide to Ga Theory , 2002 .

[30]  Jonathan N. Crook,et al.  Recent developments in consumer credit risk assessment , 2007, Eur. J. Oper. Res..

[31]  Johan A. K. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring , 2003, J. Oper. Res. Soc..