CASP9 assessment of free modeling target predictions

The goal of the biennial CASP assessment of protein structure prediction is to identify and evaluate the current state of the art methods in the field. The template free modeling (FM) category generally aims to assess ab-initio methods that predict 3D structures from a given protein sequence without the explicit use of template structures available in the Protein Data Bank. The most significant development of methodology for de novo structure prediction from sequence was introduced over a decade ago in CASP3, with the assembly of tertiary structures from selected fragments.1 Fragment-based structure assembly methods have since been adopted and developed by a number of groups, whose de novo protein structure predictions tend to outperform in CASP evaluations of the FM category.2–5 Despite this relative success, the ‘‘protein-folding problem’’ has remained unsolved, with results of the previous CASP8 FM category evaluation suggesting much room for improvement in de novo structure prediction methodologies.6 The CASP9 FM assessment included evaluating prediction models, quantifying these evaluations in a meaningful way, and using these measurements to produce group rankings that were subject to various tests of significance. Newly developed and previously applied automated methods played a crucial role in completing the assessment. Our report outlines the resulting evaluation procedure, the logic behind its development, and the results of its application to CASP9 FM target predictions. Based on these results, we highlight the progress and pitfalls of both the top performing prediction servers and the fold prediction community as a whole. The assessment of the CASP9 FM category encompassed evaluations of 30 domains, which included 4 ‘‘server only’’ domains and 26 ‘‘human/ server’’ domains. For the FM category alone, 16,971 predictions had to be evaluated, making manual judgment of all predictions impossible