Backpropagation through Time Algorithm for Training Recurrent Neural Networks using Variable Length Instances

Artificial Neural Networks (ANNs) are grouped within connectionist techniques of Artificial Intelligence. In particular, Recurrent Neural Networks are a type of ANN which is widely used in signal reproduction tasks and sequence analysis, where causal relationships in time and space take place. On the other hand, in many problems of science and engineering, signals or sequences under analysis do not always have the same length, making it difficult to select a computational technique for information processing. This article presents a flexible implementation of Recurrent Neural Networks which allows designing the desired topology based on specific application problems. Furthermore, the proposed model is capable of learning to use knowledge bases with instances of variable length in an efficient manner. The performance of the suggested implementation is evaluated through a study case of bioinformatics sequence classification. We also mention its application in obtaining artificial earthquakes from seismic scenarios similar to Cuba.

[1]  Paolo Frasconi,et al.  New Machine Learning Methods for the Prediction of Protein Topologies , 2002 .

[2]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[3]  Claudio Javier Tablada,et al.  Redes Neuronales Artificiales , 2009 .

[4]  Gonzalo Nápoles,et al.  Fuzzy Cognitive Maps for Modelling, Predicting and Interpreting HIV Drug Resistance , 2012, IBERAMIA.

[5]  Amir F. Atiya,et al.  New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  María Matilde García Lorenzo,et al.  Redes neuronales recurrentes para el análisis de secuencias , 2007 .

[7]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[8]  Gonzalo Nápoles,et al.  Modelling, Aggregation and Simulation of a Dynamic Biological System through Fuzzy Cognitive Maps , 2012, MICAI.

[9]  Thomas Lengauer,et al.  Diversity and complexity of HIV-1 drug resistance: A bioinformatics approach to predicting phenotype from genotype , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[10]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[11]  Sorin Draghici,et al.  Predicting HIV drug resistance with neural networks , 2003, Bioinform..

[12]  R. Shafer,et al.  Genotypic predictors of human immunodeficiency virus type 1 drug resistance , 2006, Proceedings of the National Academy of Sciences.

[13]  Alejandro Rosete,et al.  Combinación de clasificadores para bioinformática , 2012, Computación y Sistemas.

[14]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[15]  M. Porcel,et al.  Redes neuronales artificiales , 2001 .

[16]  Thomas Lengauer,et al.  Geno2pheno: Interpreting Genotypic HIV Drug Resistance Tests , 2001, IEEE Intell. Syst..

[17]  Thomas Villmann,et al.  Mathematical Aspects of Neural Networks , 2003, ESANN.

[18]  Barak A. Pearlmutter.  Dynamic recurrent neural networks , 1990 .

[19]  Giovanni Ulivi,et al.  Evolutionary Fuzzy Modelling for Drug Resistant HIV-1 Treatment Optimization , 2008, Engineering Evolutionary Intelligent Systems.

[20]  R. Jernigan,et al.  Self‐consistent estimation of inter‐residue protein contact energies based on an equilibrium mixture approximation of residues , 1999, Proteins.

[21]  Yvan Saeys,et al.  Predicting Human Immunodeficiency Virus (HIV) Drug Resistance Using Recurrent Neural Networks , 2006, IWINAC.

[22]  Jan Komorowski,et al.  A Rough Set-Based Model of HIV-1 Reverse Transcriptase Resistome , 2009, Bioinformatics and biology insights.

[23]  Thomas Lengauer,et al.  Data and text mining Computational methods for the design of effective therapies against drug resistant HIV strains , 2005 .

[24]  José Ramón Hilera González,et al.  Redes neuronales artificiales: fundamentos, modelos y aplicaciones , 1995 .