Interpreting Computational Neural Network Quantitative Structure-Activity Relationship Models: A Detailed Interpretation of the Weights and Biases

In this work, we present a methodology to interpret the weights and biases of a computational neural network (CNN) quantitative structure-activity relationship model. The methodology allows one to understand how an input descriptor is correlated to the predicted output by the network. The method consists of two parts. First, the nonlinear transform for a given neuron is linearized. This allows us to determine how a given neuron affects the downstream output. Next, a ranking scheme for neurons in a layer is developed. This allows us to develop interpretations of a CNN model similar in manner to the partial least squares (PLS) interpretation method for linear models described by Stanton. The method is tested on three datasets covering both physical and biological properties. The results of this interpretation method correspond well to PLS interpretations for linear models using the same descriptors as the CNN models, and they are consistent with the generally accepted physical interpretations for these properties.

[1]  Rajarshi Guha,et al.  Development of QSAR Models To Predict and Interpret the Biological Activity of Artemisinin Analogues , 2004, J. Chem. Inf. Model..

[2]  Lipo Wang,et al.  Rule extraction by genetic algorithms based on a simplified RBF neural network , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[3]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[4]  H. Ishibuchi,et al.  Fuzzy-arithmetic-based approach for extracting positive and negative linguistic rules from trained neural networks , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[5]  L B Kier,et al.  Molecular connectivity VII: specific treatment of heteroatoms. , 1976, Journal of pharmaceutical sciences.

[6]  P. Jurs,et al.  Development and use of charged partial surface area structural descriptors in computer-assisted quantitative structure-property relationship studies , 1990 .

[7]  David T. Stanton,et al.  Development and Use of Hydrophobic Surface Area (HSA) Descriptors for Computer‐Assisted Quantitative Structure—Activity and Structure—Property Relationship Studies. , 2004 .

[8]  Peter C. Jurs,et al.  Prediction of the Normal Boiling Points of Organic Compounds from Molecular Structures with a Computational Neural Network Model , 1999, J. Chem. Inf. Comput. Sci..

[9]  S. Young,et al.  Analysis of a Large Structure/Biological Activity Data Set Using Recursive Partitioning. , 2000 .

[10]  J. K. Mills,et al.  Modeling of neural networks in feedback systems using describing functions , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[11]  M Chastrette,et al.  Structure-musk odor relationships for tetralins and indans using neural networks (on the contribution of descriptors to the classification) , 1994 .

[12]  LiMin Fu,et al.  Rule Generation from Neural Networks , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[13]  A. Balaban.  Highly discriminating distance-based topological index , 1982 .

[14]  Milan Randic,et al.  On molecular identification numbers , 1984, J. Chem. Inf. Comput. Sci..

[15]  Tor Guimaraes,et al.  Integrating artificial neural networks with rule-based expert systems , 1994, Decis. Support Syst..

[16]  Rajarshi Guha,et al.  Interpreting Computational Neural Network QSAR Models: A Measure of Descriptor Importance , 2005, J. Chem. Inf. Model..

[17]  Mark T D Cronin,et al.  Quantitative structure-activity relationships (QSARs) for the prediction of skin permeation of exogenous chemicals. , 2002, Chemosphere.

[18]  Harpreet S. Chadha,et al.  Molecular Factors Influencing Drug Transfer across the Blood‐Brain Barrier , 1997, The Journal of pharmacy and pharmacology.

[19]  Donald W. Miller,et al.  Brain uptake of drugs : the influence of chemical and biological factors , 1992 .

[20]  David T. Stanton,et al.  On the Physical Interpretation of QSAR Models , 2003, J. Chem. Inf. Comput. Sci..

[21]  Susu Yao,et al.  Evolving fuzzy neural networks for extracting rules , 1996, Proceedings of IEEE 5th International Fuzzy Systems.

[22]  L B Kier,et al.  Molecular connectivity. I: Relationship to nonspecific local anesthesia. , 1975, Journal of pharmaceutical sciences.

[23]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[24]  H. Tsukimoto,et al.  Rule extraction from neural networks via decision tree induction , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[25]  Simon Urbanek.  EXPLORING STATISTICAL FORESTS , 2002 .

[26]  G. David Garson,et al.  Interpreting neural-network connection weights , 1991 .

[27]  Peter C. Jurs,et al.  Automated Descriptor Selection for Quantitative Structure-Activity Relationships Using Generalized Simulated Annealing , 1995, J. Chem. Inf. Comput. Sci..

[28]  Ross F. Hayward,et al.  The truth is in there: current issues in extracting rules from trained feedforward artificial neural networks , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[29]  E. George,et al.  Making sense of a forest of treesH , 2007 .

[30]  José Manuel Benítez,et al.  Interpretation of artificial neural networks by means of fuzzy rules , 2002, IEEE Trans. Neural Networks.

[31]  Robert W. Blanning,et al.  An empirical measure of element contribution in neural networks , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[32]  P. Jurs,et al.  Development of Linear, Ensemble, and Nonlinear Models for the Prediction and Interpretation of the Biological Activity of a Set of PDGFR Inhibitors. , 2005 .

[33]  César Hervás-Martínez,et al.  Heuristic Extraction of Rules in Pruned Artificial Neural Networks Models Used for Quantifying Highly Overlapping Chromatographic Peaks , 2004, J. Chem. Inf. Model..

[34]  Thomas Kailath,et al.  Rational approximation, harmonic analysis and neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[35]  Kurt Hornik,et al.  Some new results on neural network approximation , 1993, Neural Networks.

[36]  Amit Gupta,et al.  Generalized Analytic Rule Extraction for Feedforward Neural Networks , 1999, IEEE Trans. Knowl. Data Eng..

[37]  Guido Bologna.  Rule extraction from linear combinations of DIMLP neural networks , 2000, Proceedings. Vol.1. Sixth Brazilian Symposium on Neural Networks.