q-fin . P M ] 10 O ct 2 00 8 LookAhead Benchmark Bias in Portfolio Performance Evaluation ∗

Performance of investment managers is predominantly evaluated against targeted benchmarks, such as stock, bond, or commodity indices. But most professional databases do not retain time series for companies that drop from the database and do not necessarily track changes in the benchmarks over time. Consequently, standard tests of portfolio performance suffer from look-ahead benchmark bias, meaning that a given strategy is naively backtested against the assets constituting the benchmark of reference at the end of the testing period (i.e., now), rather than at the beginning of the period. In this article, the authors report that look-ahead benchmark bias can be surprisingly large in portfolios of common stocks—up to 8% per annum when the S&P 500 Index is the benchmark. Using CRSP data for the running top 500 U.S. capitalizations over the period 1927–2006, the authors demonstrate that look-ahead benchmark bias can account for a gross overestimation in performance metrics, such as the Sharpe ratio, as well as an underestimation of risk measured, for example, by peak-to-valley drawdowns. A general methodology to test investment strategy properties is advanced by the authors in the context of several random strategies having similar investment constraints.

[1]  T. Jordan Earthquake Predictability, Brick by Brick , 2006 .

[2]  Tilburg University Survival , LookAhead Bias and the Persistence in Hedge Fund Performance , 2002 .

[3]  Marno Verbeek,et al.  Survival, Look-Ahead Bias and the Persistence in Hedge Fund Performance , 2002 .

[4]  Stefano Scarpetta,et al.  Comparative analysis of firm demographics and survival: micro-level evidence for the OECD countries , 2003 .

[5]  Patrick Burns Random Portfolios for Evaluating Trading Strategies , 2006 .

[6]  Eliminating lookahead bias in evaluating persistence in mutual fund performance q , 2001 .

[7]  O. Scaillet,et al.  False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas , 2005 .

[8]  A RossStephen Survivorship Bias in Performance Studies , 2009 .

[9]  Christopher R. Blake,et al.  Survivorship Bias and Mutual Fund Performance , 1995 .

[10]  David K. Musto,et al.  Mutual Fund Survivorship , 1997 .

[11]  William N. Goetzmann,et al.  Offshore Hedge Funds: Survival and Performance 1989-1995 , 1997 .

[12]  S. Wiemer,et al.  Earthquake Likelihood Model Testing , 2007 .

[13]  A. Lo,et al.  Data-Snooping Biases in Tests of Financial Asset Pricing Models , 1989 .

[14]  Raymond Kan,et al.  Optimal Portfolio Choice with Parameter Uncertainty , 2007, Journal of Financial and Quantitative Analysis.

[15]  Marno Verbeek,et al.  Eliminating look-ahead bias in evaluating persistence in mutual fund performance , 2001 .

[16]  Michael Wolf,et al.  Centre De Referència En Economia Analítica Barcelona Economics Working Paper Series Working Paper Nº 17 Stewise Multiple Testing as Formalized Data Snooping Stepwise Multiple Testing as Formalized Data Snooping , 2022 .

[17]  H. White,et al.  Data‐Snooping, Technical Trading Rule Performance, and the Bootstrap , 1999 .

[18]  H. White,et al.  A Reality Check for Data Snooping , 2000 .

[19]  Marno Verbeek,et al.  Fund Liquidation, Self-Selection, and Look-Ahead Bias in the Hedge Fund Industry , 2004 .

[20]  Amy E. Knaup Survival and Longevity in the Business Employment Dynamics Data , 2005 .

[21]  R. Zeckhauser,et al.  The J-Shape of Performance Persistence Given Survivorship Bias , 1997, Review of Economics and Statistics.

[22]  D Sornette,et al.  Algorithm for model validation: Theory and applications , 2007, Proceedings of the National Academy of Sciences.

[23]  A. Lynch,et al.  Survivorship Bias and Attrition Effects in Measures of Performance Persistence , 1998 .

[24]  Sheridan Titman,et al.  On Persistence in Mutual Fund Performance , 1997 .

[25]  Rejoinder: The J-Shape of Performance Persistence Given Survivorship Bias , 1997, Review of Economics and Statistics.

[26]  Jie Cai,et al.  Index Rebalancing and Long-Term Portfolio Performance , 2007 .