Supervisory Stress Tests, Model Risk, and Model Disclosure: Lessons from OFHEO

The Federal Reserve has recently embraced stress testing as an important component of its supervisory program for the very largest U.S. banking organizations. While stress tests can provide valuable insights, they are vulnerable to model risk. This paper explores the issue of model risk through a case study of a recent U.S. supervisory experience with a complex and fully disclosed stress test that failed spectacularly: OFHEO’s risk-based capital stress test for Fannie Mae and Freddie Mac. Our analysis focuses on a key element of OFHEO’s stress test: the performance of 30-year fixed-rate mortgages. After illustrating the poor default and prepayment forecasting performance of the model as implemented, we find that the primary explanation for this model failure was that OFHEO never re-estimated the model and hence left parameters static for almost a decade. We also show that default forecast performance would have been enhanced by including additional variables as the market evolved during the 2000s, like credit scores, documentation levels, vintage effects, and more disaggregated house price indices. Finally, we demonstrate that the house price stress inherent in the OFHEO model was significantly less stressful than the actual recent U.S. experience. Taken together, we believe that our results demonstrate the economic importance of model risk in stress testing and should lend support to efforts to mitigate such risk through continuous model development and independent validation. JEL Classifications: G21, G23, G28