An introduction to recursive neural networks and kernel methods for cheminformatics.

The aim of this paper is to introduce the reader to new developments in Neural Networks and Kernel Machines concerning the treatment of structured domains. Specifically, we discuss the research on these relatively new models to introduce a novel and more general approach to QSPR/QSAR analysis. The focus is on the computational side and not on the experimental one.

[1]  Alessio Micheli,et al.  Recursive Processing of Structured Domains in Machine Learning -PhD Thesis, TD-13/03 , 2003 .

[2]  A. Starita,et al.  Fisher kernel for tree structured data , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[3]  Peter L. Bartlett,et al.  The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important than the Size of the Network , 1998, IEEE Trans. Inf. Theory.

[4]  Scott E. Fahlman,et al.  The Recurrent Cascade-Correlation Architecture , 1990, NIPS.

[5]  Alexander J. Smola,et al.  Fast Kernels for String and Tree Matching , 2002, NIPS.

[6]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[7]  Alessandro Sperduti,et al.  A general framework for adaptive processing of data structures , 1998, IEEE Trans. Neural Networks.

[8]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[9]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[10]  Alessio Ceroni,et al.  Learning protein secondary structure from sequential and relational data , 2005, Neural Networks.

[11]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[12]  Alessio Micheli,et al.  A New Neural Network Model for Contextual Processing of Graphs , 2005, WIRN/NAIS.

[13]  Alessandro Sperduti,et al.  Extended Cascade-Correlation for Syntactic and Structural Pattern Recognition , 1996, SSPR.

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  Thomas Gärtner,et al.  On Graph Kernels: Hardness Results and Efficient Alternatives , 2003, COLT.

[16]  Thomas Hofmann,et al.  Support vector machine learning for interdependent and structured output spaces , 2004, ICML.

[17]  Thomas Gärtner,et al.  A survey of kernels for structured data , 2003, SKDD.

[18]  Franco Scarselli,et al.  Recursive processing of cyclic graphs , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[19]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[20]  Ashwin Srinivasan,et al.  Relating chemical activity to structure: An examination of ILP successes , 1995, New Generation Computing.

[21]  Alessio Micheli,et al.  Universal Approximation Capability of Cascade Correlation for Structures , 2005, Neural Computation.

[22]  Michael I. Jordan.  Graphical Models , 2003 .

[23]  Thomas Gärtner,et al.  Cyclic pattern kernels for predictive graph mining , 2004, KDD.

[24]  Pierre Baldi,et al.  Graph kernels for chemical informatics , 2005, Neural Networks.

[25]  Jason Weston,et al.  Mismatch String Kernels for SVM Protein Classification , 2002, NIPS.

[26]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[27]  Alessio Micheli,et al.  Recursive cascade correlation for contextual processing of structured data , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[28]  Frank R. Burden,et al.  Use of Automatic Relevance Determination in QSAR Studies Using Bayesian Neural Networks , 2000, J. Chem. Inf. Comput. Sci..

[29]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[30]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[31]  Pierre Baldi,et al.  Bioinformatics - the machine learning approach (2. ed.) , 2000 .

[32]  Pierre Baldi,et al.  The Principled Design of Large-Scale Recursive Neural Network Architectures--DAG-RNNs and the Protein Structure Prediction Problem , 2003, J. Mach. Learn. Res..

[33]  Barbara Hammer,et al.  Learning with recurrent neural networks , 2000 .

[34]  Y. Freund,et al.  Profile-based string kernels for remote homology detection and motif extraction. , 2005, Journal of bioinformatics and computational biology.

[35]  David Weininger,et al.  SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules , 1988, J. Chem. Inf. Comput. Sci..

[36]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[37]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[38]  Giovanni Soda,et al.  Exploiting the past and the future in protein secondary structure prediction , 1999, Bioinform..

[39]  David G. Stork,et al.  Pattern Classification , 1973 .

[40]  Jean-Philippe Vert,et al.  The context-tree kernel for strings , 2005, Neural Networks.

[41]  Paolo Frasconi,et al.  Weighted decomposition kernels , 2005, ICML.

[42]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[43]  Tatsuya Akutsu,et al.  Protein homology detection using string alignment kernels , 2004, Bioinform..

[44]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[45]  Lonnie Chrisman,et al.  Learning Recursive Distributed Representations for Holistic Computation , 1991 .

[46]  C. Hansch,et al.  Quantitative Structure‐Activity Relationships of the Benzodiazepines. A Review and Reevaluation. , 1995 .

[47]  Alessio Micheli,et al.  Application of Cascade Correlation Networks for Structures to Chemistry , 2004, Applied Intelligence.

[48]  Alessio Micheli,et al.  Predicting Thermodynamic Properties from Molecular Structures by Recursive Neural Networks , 2004 .

[49]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[50]  Alessio Micheli,et al.  Contextual processing of structured data by recursive cascade correlation , 2004, IEEE Transactions on Neural Networks.

[51]  Tamás Horváth,et al.  Cyclic Pattern Kernels Revisited , 2005, PAKDD.

[52]  Alessandro Sperduti,et al.  Special issue on neural networks and kernel methods for structured domains , 2005, Neural Networks.

[53]  Alessio Micheli,et al.  Analysis of the Internal Representations Developed by Neural Networks for Structures Applied to Quantitative Structure-Activity Relationship Studies of Benzodiazepines , 2001, J. Chem. Inf. Comput. Sci..

[54]  Ashwin Srinivasan,et al.  Theories for Mutagenicity: A Study in First-Order and Feature-Based Induction , 1996, Artif. Intell..

[55]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[56]  Alessio Micheli,et al.  Quantitative structure-activity relationships of Benzodiazepines by recursive cascade correlation , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[57]  Alessio Micheli,et al.  A preliminary empirical comparison of recursive neural networks and tree kernel methods on regression tasks for tree structured domains , 2005, Neurocomputing.

[58]  Paolo Frasconi,et al.  Disulfide connectivity prediction using recursive neural networks and evolutionary information , 2004, Bioinform..

[59]  Alessio Micheli,et al.  Prediction of Chemical-Phisical Properties by Recursive Neural Networks for Structures , 2005 .

[60]  Alessio Micheli,et al.  Design of new biologically active molecules by recursive neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[61]  J. Rigaudy,et al.  Nomenclature of organic chemistry , 1979 .

[62]  David Haussler,et al.  A Discriminative Framework for Detecting Remote Protein Homologies , 2000, J. Comput. Biol..

[63]  Tatsuya Akutsu,et al.  Extensions of marginalized graph kernels , 2004, ICML.

[64]  Pierre Baldi,et al.  Kernels for small molecules and the prediction of mutagenicity, toxicity and anti-cancer activity , 2005, ISMB.

[65]  Michael Collins,et al.  Convolution Kernels for Natural Language , 2001, NIPS.

[66]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[67]  Bernhard Schölkopf,et al.  Kernel Methods in Computational Biology , 2005 .

[68]  Thomas G. Dietterich,et al.  Editors. Advances in Neural Information Processing Systems , 2002 .

[69]  Alessio Micheli,et al.  Bi-causal recurrent cascade correlation , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[70]  Paolo Frasconi,et al.  Prediction of Protein Topologies Using GIOHMMs and GRNNs , 2003, NIPS 2003.

[71]  Alessio Micheli,et al.  Dealing with Graphs by Neural Networks , 2005 .

[72]  S. Džeroski,et al.  Relational Data Mining , 2001, Springer Berlin Heidelberg.

[73]  David Haussler,et al.  Exploiting Generative Models in Discriminative Classifiers , 1998, NIPS.

[74]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[75]  David Haussler,et al.  Using the Fisher Kernel Method to Detect Remote Protein Homologies , 1999, ISMB.

[76]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[77]  D. Villemin,et al.  Use of a neural network to determine the boiling point of alkanes , 1994 .

[78]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .