Incorporating knowledge in multi-layer networks: the example of protein secondary structure prediction

We present on the example of protein secondary structure prediction various ways by which domain specific knowledge can be incorporated into a multi-layer network so as to increase speed of learning and accuracy in prediction. In particular, we show how to set weight patterns so as to reproduce knowledge of the domain. We illustrate the use of linear learning and Discriminant Analysis to specify the appropriate number of hidden units. Finally, we show that “guided” pruning can improve accuracy while reducing the number of weights. These various techniques can be used for other applications as well.

[1]  Barry Robson,et al.  An algorithm for secondary structure determination in proteins based on sequence similarity , 1986, FEBS letters.

[2]  Patrick Gallinari,et al.  EVALUATION OF NETWORK ARCHITECTURES ON TEST LEARNING TASKS. , 1987 .

[3]  T. Sejnowski,et al.  Predicting the secondary structure of globular proteins using neural network models. , 1988, Journal of molecular biology.

[4]  Yann LeCun,et al.  Improving the convergence of back-propagation learning with second-order methods , 1989 .

[5]  N. Sharkey,et al.  Advances in cognitive science , 1988 .

[6]  W. Kabsch,et al.  Dictionary of protein secondary structure: Pattern recognition of hydrogen‐bonded and geometrical features , 1983, Biopolymers.

[7]  P. Y. Chou,et al.  Empirical predictions of protein conformation. , 1978, Annual review of biochemistry.

[8]  R. Pfeifer,et al.  Connectionism in Perspective , 1989 .

[9]  V. Lim.  Algorithms for prediction of α-helical and β-structural regions in globular proteins , 1974 .

[10]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[11]  Y. L. Cun,et al.  Modèles connexionnistes de l'apprentissage , 1987 .

[12]  Françoise Fogelman-Soulié,et al.  Speaker-independent isolated digit recognition: Multilayer perceptrons vs. Dynamic time warping , 1990, Neural Networks.

[13]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[14]  N. Meyers,et al.  H = W. , 1964, Proceedings of the National Academy of Sciences of the United States of America.

[15]  Yann LeCun,et al.  Generalization and network design strategies , 1989 .

[16]  Raymond L. Watrous.  Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization , 1988 .

[17]  Patrick Gallinari,et al.  Multilayer perceptrons and data analysis , 1988, IEEE 1988 International Conference on Neural Networks.

[18]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[19]  M. Karplus,et al.  Protein secondary structure prediction with a neural network. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Olivier Gascuel,et al.  A simple method for predicting the secondary structure of globular proteins: implications and accuracy , 1988, Comput. Appl. Biosci..

[21]  J. Garnier,et al.  Analysis of the accuracy and implications of simple methods for predicting the secondary structure of globular proteins. , 1978, Journal of molecular biology.

[22]  W. Kabsch,et al.  How good are predictions of protein secondary structure? , 1983, FEBS letters.

[23]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[24]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.