WHAT IS A MORE POWERFUL MODEL WORTH ?

In this paper, we provide empirical evidence of the economic impact of differences in power between various default models. We find that this value can vary substantially, and that in general more powerful models lead to economically significant differences in portfolio performance. In one case, for example, we found, for a prototypical ($50B) bank, that using a model with a five point higher accuracy ratio could result in increased profitability of $2.1MM to $4.8MM per year. A companion paper, Stein (2003), presented a general framework for defining optimal lending cutoffs. It turns out that the framework for defining cutoffs can be extended to a more complete pricing approach that is more flexible and this was shown in that paper. In both cases, the framework permits the evaluation of models in terms of economic benefits and demonstrates that more powerful models are generally more cost effective than weaker ones. However, due in part to the non-parametric nature of power curves as well as the heterogeneity of the middle market banking environment, it can be difficult to size the magnitude of this economic value analytically. In this paper, we show results of simulations based on an analysis of CRD database of middle-market financial statement and loan performance data. We simulated lending using default prediction models of various qualities. We examined both the cutoffand pricing-based lending approaches. In both cases we found that the use of a more powerful model, on average, leads to substantial economic benefit over that of a weaker model. We also provide case studies giving some insight into why the more powerful models are preferred. We discuss examples of dollar values of typical expected differentials in profit for prototypical small, medium and large institutions at various power levels. The methodology we present is a general one and can be used to size the financial value of lending policies based on any many varying sets of models. AUTHOR Roger M.Stein Felipe Jordão 1 We have extended our research on the economic value of default models since the original posting of this paper. The new research primarily involved a larger number of default models and the exploration of a number of robust methods for sizing the statistical relationship between the simulation results on differences in power and economic performance. The updated results are presented in this version of the paper. © 2003 Moody’s KMV Company. All rights reserved. Credit Monitor®, EDFCalc®, Private Firm Model®, KMV®, CreditEdge, Portfolio Manager, Portfolio Preprocessor, GCorr, DealAnalyzer, CreditMark, the KMV logo, Moody's RiskCalc, Moody's Financial Analyst, Moody's Risk Advisor, LossCalc, Expected Default Frequency, and EDF are trademarks of MIS Quality Management Corp. ACKNOWLEDGEMENTS We are grateful to Jalal Akhavein, Ivo Antonov, Jeff Bohn, Greg Gupton and Ahmet Kocagil of Moody's KMV for their generous comments. Jon DiGiambattista, Brian Asparro, and Ralf Imming provided invaluable expertise in the compilation and analysis of the data for several of these experiments. All remaining errors are of course our own. Published by: Moody’s KMV Company To Learn More Please contact your Moody’s KMV client representative, visit us online at www.moodyskmv.com, contact Moody’s KMV via e-mail at info@mkmv.com, or call us at: NORTH AND SOUTH AMERICA, NEW ZEALAND AND AUSTRALIA, CALL: 1 866 321 MKMV (6568) or 415 296 9669 EUROPE, THE MIDDLE EAST, AFRICA AND INDIA, CALL: 44 20 7778 7400 FROM ASIA CALL: 813 3218 116