Evaluation of model quality predictions in CASP9

CASP has been assessing the state of the art in the a priori estimation of accuracy of protein structure prediction since 2006. The inclusion of model quality assessment category in CASP contributed to a rapid development of methods in this area. In the last experiment, 46 quality assessment groups tested their approaches to estimate the accuracy of protein models as a whole and/or on a per‐residue basis. We assessed the performance of these methods predominantly on the basis of the correlation between the predicted and observed quality of the models on both global and local scales. The ability of the methods to identify the models closest to the best one, to differentiate between good and bad models, and to identify well modeled regions was also analyzed. Our evaluations demonstrate that even though global quality assessment methods seem to approach perfection point (weighted average per‐target Pearson's correlation coefficients are as high as 0.97 for the best groups), there is still room for improvement. First, all top‐performing methods use consensus approaches to generate quality estimates, and this strategy has its own limitations. Second, the methods that are based on the analysis of individual models lag far behind clustering techniques and need a boost in performance. The methods for estimating per‐residue accuracy of models are less accurate than global quality assessment methods, with an average weighted per‐model correlation coefficient in the range of 0.63–0.72 for the best 10 groups. Proteins 2011; © 2011 Wiley‐Liss, Inc.