GRID-Based Three-Dimensional Pharmacophores II: PharmBench, a Benchmark Data Set for Evaluating Pharmacophore Elucidation Methods

To date, published pharmacophore elucidation approaches typically use a handful of data sets for validation: here, we have assembled a data set for 81 targets, containing 960 ligands aligned using their cocrystallized protein targets, to provide the experimental "gold standard". The two-dimensional structures are also assembled to remove conformational bias; an ideal method would be able to take these structures as input, find the common features, and reproduce the bioactive conformations and their alignments to correspond with the X-ray-determined gold standard alignments. Here we present this data set and describe three objective measures to evaluate performance: the ability to identify the bioactive conformation, the ability to identify and correctly align this conformation for 50% of the molecules in each data set, and the pharmacophoric field similarity. We have applied this validation methodology to our pharmacophore elucidation method FLAPpharm, that is published in the first paper of this series and discuss the limitations of the data set and objective success criteria. Starting from two-dimensional structures and producing unbiased models, FLAPpharm was able to identify the bioactive conformations for 67% of the ligands and also to produce successful models according to the second metric for 67% of the Pharmbench data sets. Inspection of the unsuccessful models highlighted the limitation of this root mean square (rms)-derived metric, since many were found to be pharmacophorically reasonable, increasing the overall success rate to 83%. The PharmBench data set is available at http://www.moldiscovery.com/PharmBench , along with a web service to enable users to score model alignments coming from external methods in the same way that we have presented here and, therefore, establishes a pharmacophore elucidation benchmark data set available to be used by the community.

[1]  Stefano Alcaro,et al.  Computational analysis of Human Immunodeficiency Virus (HIV) Type-1 reverse transcriptase crystallographic models based on significant conserved residues found in Highly Active Antiretroviral Therapy (HAART)-treated patients. , 2010, Current medicinal chemistry.

[2]  J. Irwin,et al.  Benchmarking sets for molecular docking. , 2006, Journal of medicinal chemistry.

[3]  Hans Matter,et al.  Structural requirements for factor Xa inhibition by 3-oxybenzamides with neutral P1 substituents: combining X-ray crystallography, 3D-QSAR, and tailored scoring functions. , 2005, Journal of medicinal chemistry.

[4]  Paolo Benedetti,et al.  FLAP: GRID Molecular Interaction Fields in Virtual Screening. Validation using the DUD Data Set , 2010, J. Chem. Inf. Model..

[5]  Andrew R. Leach,et al.  A comparison of the pharmacophore identification programs: Catalyst, DISCO and GASP , 2002, J. Comput. Aided Mol. Des..

[6]  Stefano Alcaro,et al.  GBPM: GRID-based pharmacophore model: concept and application studies to protein-protein recognition , 2006, Bioinform..

[7]  Jun Feng,et al.  PharmID: Pharmacophore Identification Using Gibbs Sampling , 2006, J. Chem. Inf. Model..

[8]  P. Goodford.  A computational procedure for determining energetically favorable binding sites on biologically important macromolecules. , 1985, Journal of medicinal chemistry.

[9]  Richard A. Lewis,et al.  Three-dimensional pharmacophore methods in drug discovery. , 2010, Journal of medicinal chemistry.

[10]  Gareth Jones,et al.  GAPE: An Improved Genetic Algorithm for Pharmacophore Elucidation , 2010, J. Chem. Inf. Model..

[11]  Andrew Smellie,et al.  Identification of Common Functional Configurations Among Molecules , 1996, J. Chem. Inf. Comput. Sci..

[12]  Gareth Jones,et al.  A genetic algorithm for flexible molecular overlay and pharmacophore elucidation , 1995, J. Comput. Aided Mol. Des..

[13]  Thierry Langer,et al.  LigandScout: 3-D Pharmacophores Derived from Protein-Bound Ligands and Their Use as Virtual Screening Filters , 2005, J. Chem. Inf. Model..

[14]  David E. Shaw,et al.  PHASE: a new engine for pharmacophore perception, 3D QSAR model development, and 3D database screening: 1. Methodology and preliminary results , 2006, J. Comput. Aided Mol. Des..

[15]  Tudor I. Oprea,et al.  Optimization of CAMD techniques 3. Virtual screening enrichment studies: a help or hindrance in tool selection? , 2008, J. Comput. Aided Mol. Des..

[16]  P E Bourne,et al.  Protein structure alignment by incremental combinatorial extension (CE) of the optimal path. , 1998, Protein engineering.

[17]  Valerie J. Gillet,et al.  Multiobjective Optimization of Pharmacophore Hypotheses: Bias Toward Low-Energy Conformations , 2009, J. Chem. Inf. Model..

[18]  Thierry Langer,et al.  Efficient overlay of small organic molecules using 3D pharmacophores , 2007, J. Comput. Aided Mol. Des..

[19]  Gabriele Cruciani,et al.  A Common Reference Framework for Analyzing/Comparing Proteins and Ligands. Fingerprints for Ligands And Proteins (FLAP): Theory and Application , 2007, J. Chem. Inf. Model..

[20]  Simon Cross,et al.  GRID-Based Three-Dimensional Pharmacophores I: FLAPpharm, a Novel Approach for Pharmacophore Elucidation , 2012, J. Chem. Inf. Model..