Alternative Global Goodness Metrics and Sensitivity Analysis: Heuristics to Check the Robustness of Conclusions from Studies Comparing Virtual Screening Methods

We introduce two ways of testing the robustness of conclusions from studies comparing virtual screening methods: alternative "global goodness" metrics and sensitivity analysis. While the robustness tests cannot eliminate all biases in virtual screening comparisons, they are useful as a "reality check" for any given study. To illustrate this, we apply them to a set of enrichments published in McGaughey et al. (J. Chem. Inf. Model. 2007, 47, 1504-1519) where 11 target protein/ligand combinations are tested on 2D and 3D similarity methods, plus docking. The major conclusions in that paper, for instance, that ligand-based methods are better than docking methods, hold up. However, some minor conclusions, such as Glide being the best docking method, do not.

[1]  J. Jenkins,et al.  A 3D similarity method for scaffold hopping from known drugs or natural ligands to new chemotypes. , 2004, Journal of medicinal chemistry.

[2]  Thierry Kogej,et al.  Multifingerprint Based Similarity Searches for Targeted Class Compound Selection , 2006, J. Chem. Inf. Model..

[3]  Andrew C. Good,et al.  Measuring CAMD technique performance: A virtual screening case study in the design of validation experiments , 2004, J. Comput. Aided Mol. Des..

[4]  Didier Rognan,et al.  Comparative evaluation of eight docking tools for docking and virtual screening accuracy , 2004, Proteins.

[5]  Kenji Onodera,et al.  Evaluations of Molecular Docking Programs for Virtual Screening , 2007, J. Chem. Inf. Model..

[6]  Christopher I. Bayly,et al.  Evaluating Virtual Screening Methods: Good and Bad Metrics for the "Early Recognition" Problem , 2007, J. Chem. Inf. Model..

[7]  Robert P. Sheridan,et al.  Comparison of Topological, Shape, and Docking Methods in Virtual Screening. , 2007 .

[8]  C. David Andersson,et al.  A Multivariate Approach to Investigate Docking Parameters' Effects on Docking Performance , 2007, J. Chem. Inf. Model..

[9]  Maria Kontoyianni,et al.  Evaluation of library ranking efficacy in virtual screening , 2005, J. Comput. Chem..

[10]  Jin Li,et al.  On Evaluating Molecular-Docking Methods for Pose Prediction and Enrichment Factors , 2006, J. Chem. Inf. Model..

[11]  Olivier Sperandio,et al.  MED-SuMoLig: A New Ligand-Based Screening Tool for Efficient Scaffold Hopping , 2007, J. Chem. Inf. Model..

[12]  Robert P. Sheridan,et al.  Chemical Similarity Using Physiochemical Property Descriptors , 1996, J. Chem. Inf. Comput. Sci..

[13]  Paul N. Mortenson,et al.  Diverse, high-quality test set for the validation of protein-ligand docking performance. , 2007, Journal of medicinal chemistry.

[14]  E. Fluder,et al.  Protocols for Bridging the Peptide to Nonpeptide Gap in Topological Similarity Searches. , 2001 .

[15]  Magdalena Bacilieri,et al.  Combining ligand-based and structure-based drug design in the virtual screening arena , 2007, Expert opinion on drug discovery.

[16]  Ramaswamy Nilakantan,et al.  Topological torsion: a new molecular descriptor for SAR applications. Comparison with other descriptors , 1987, J. Chem. Inf. Comput. Sci..

[17]  R. Venkataraghavan,et al.  Atom pairs as molecular features in structure-activity studies: definition and applications , 1985, J. Chem. Inf. Comput. Sci..

[18]  J. Pin,et al.  Virtual screening workflow development guided by the "receiver operating characteristic" curve approach. Application to high-throughput docking on metabotropic glutamate receptor subtype 4. , 2005, Journal of medicinal chemistry.

[19]  Richard A. Friesner,et al.  Comparative Performance of Several Flexible Docking Programs and Scoring Functions: Enrichment Studies for a Diverse Set of Pharmaceutically Relevant Targets. , 2007 .

[20]  David Vidal,et al.  LINGO, an Efficient Holographic Text Based Method To Calculate Biophysical Properties and Intermolecular Similarities , 2005, J. Chem. Inf. Model..

[21]  W Patrick Walters,et al.  A detailed comparison of current docking and scoring methods on systems of pharmaceutical relevance , 2004, Proteins.

[22]  Qiang Zhang,et al.  Scaffold hopping through virtual screening using 2D and 3D similarity descriptors: ranking, voting, and consensus scoring. , 2006, Journal of medicinal chemistry.

[23]  P. Hawkins,et al.  Comparison of shape-matching and docking as virtual screening tools. , 2007, Journal of medicinal chemistry.

[24]  C. E. Peishoff,et al.  A critical assessment of docking programs and scoring functions. , 2006, Journal of medicinal chemistry.

[25]  Markus H. J. Seifert.  Assessing the Discriminatory Power of Scoring Functions for Virtual Screening , 2006, J. Chem. Inf. Model..

[26]  E. Jaeger,et al.  Comparison of automated docking programs as virtual screening tools. , 2005, Journal of Medicinal Chemistry.

[27]  J. Irwin,et al.  Benchmarking sets for molecular docking. , 2006, Journal of medicinal chemistry.

[28]  Miklos Feher,et al.  Novel 2D Fingerprints for Ligand-Based Virtual Screening , 2006, J. Chem. Inf. Model..