The distribution of defaults and Bayesian model validation

Quantitative rating systems are increasingly being used for the purposes of capital allocation and pricing credits. For these purposes, it is important to validate the accuracy of the probability of default (PD) estimates generated by the rating system and not merely focus on evaluating the discriminatory power of the system. The validation of the accuracy of the PD quantification has been a challenge, fraught with theoretical difficulties (mainly, the impact of correlation) and data issues (e.g., the infrequency of default events). Moreover, models— even ‘correct’ models—will over-predict default rates most of the time. AUTHOR Douglas W. Dwyer This paper addresses the challenge by presenting a means of assessing when the realized default rate will differ from the expected default rate. We first set up the standard single factor framework for determining the distribution of defaults that one would expect in a bucket of exposures which, according to the rating system output, have a uniform PD. We then extend the framework to analyze the distribution of defaults across multiple buckets and the distribution of defaults in one bucket over several time periods. Based on this framework, we present two Bayesian approaches in relation to two common modeling issues which occur in practice. The first approach provides a set of techniques to facilitate risk assessment in the absence of sufficient historical default data. It allows us to determine the posterior distribution of a PD, given zero realized defaults, thereby providing a framework for determining the upper bound for a PD in relation to a low default portfolio. The second approach provides a means for monitoring the calibration of a rating system. It allows us to determine the posterior distribution of the aggregate shock in the macroeconomic environment, given a realized default rate. By comparing this distribution to the institution’s view of the stage of the credit cycle its borrowers are in, this approach provides useful insight for whether an institution should revisit the calibration of its rating system. This method allows one to determine that a calibration needs to be revisited even when the default rate is within the 95% confidence level computed under the standard approach. Copyright© 2006, Moody’s KMV Company. All rights reserved. Credit Monitor, CreditEdge, CreditEdge Plus, CreditMark, DealAnalyzer, EDFCalc, Private Firm Model, Portfolio Preprocessor, GCorr, the Moody’s KMV logo, Moody’s KMV Financial Analyst, Moody’s KMV LossCalc, Moody’s KMV Portfolio Manager, Moody’s KMV Risk Advisor, Moody’s KMV RiskCalc, RiskAnalyst, Expected Default Frequency, and EDF are trademarks owned by of MIS Quality Management Corp. and used under license by Moody’s KMV Company. ACKNOWLEDGEMENTS I am grateful to Joseph McQuown, Bill Morokoff, Martha Sellers, Jamie Stark, Roger Stein, Kenneth Wee, Sarah Woo and Andrew Zhang for helpful comments and suggestions. All remaining errors are, of course, my own. Published by: Moody’s KMV Company To Learn More Please contact your Moody’s KMV client representative, visit us online at www.moodyskmv.com, contact Moody’s KMV via e-mail at info@mkmv.com, or call us at: FROM NORTH AND SOUTH AMERICA CALL: 1 866 321 MKMV (6568) or 415 874 6000 FROM EUROPE, THE MIDDLE EAST, AND AFRICA CALL: 44 20 7280 8300 FROM ASIA, NEW ZEALAND, AUSTRALIA AND INDIA CALL: 813 3218 1160