Critical assessment of protein intrinsic disorder prediction

Intrinsically disordered proteins defying the traditional protein structure-function paradigm represent a challenge to study experimentally. As a large part of our knowledge rests on computational predictions, it is crucial for their accuracy to be high. The Critical Assessment of protein Intrinsic Disorder prediction (CAID) experiment was established as a community-based blind test to determine the state of the art in predicting intrinsically disordered regions in proteins and the subset of disordered residues involved in binding other molecules. A total of 43 methods, 32 for disorder and 11 for binding regions, were evaluated on a dataset of 646 novel manually curated proteins from DisProt. The best methods use deep learning techniques and significantly outperform widely used earlier physicochemical methods across different types of targets. Disordered binding regions remain hard to predict correctly. Depending on the definition used, the top disorder predictor has an FMax of 0.483 (DisProt) or 0.792 (DisProt-PDB). As the top binding predictor only attains an FMax of 0.231, this suggests significant potential for improvement. Intriguingly, computing times among the top performing methods vary by up to four orders of magnitude.