SCALED SELF-ORGANIZING MAP – HIDDEN MARKOV MODEL ARCHITECTURE FOR BIOLOGICAL SEQUENCE CLUSTERING

The self-organizing hidden Markov model map (SOHMMM) introduces a hybrid integration of the self-organizing map (SOM) and the hidden Markov model (HMM). Its scaled, online gradient descent unsupervised learning algorithm is an amalgam of the SOM unsupervised training and the HMM reparameterized forward-backward techniques. In essence, with each neuron of the SOHMMM lattice, an HMM is associated. The image of an input sequence on the SOHMMM mesh is defined as the location of the best matching reference HMM. Model tuning and adaptation can take place directly from raw data, within an automated context. The SOHMMM can accommodate and analyze deoxyribonucleic acid, ribonucleic acid, protein chain molecules, and generic sequences of high dimensionality and variable lengths encoded directly in nonnumerical/symbolic alphabets. Furthermore, the SOHMMM is capable of integrating and exploiting latent information hidden in the spatiotemporal dependencies/correlations of sequences’ elements.

[1]  Andreas Stafylopatis,et al.  Sequence clustering with the Self-Organizing Hidden Markov Model Map , 2008, 2008 8th IEEE International Conference on BioInformatics and BioEngineering.

[2]  Thomas Voegtlin,et al.  Recursive self-organizing maps , 2002, Neural Networks.

[3]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[4]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[5]  Younès Bennani,et al.  LEARNING SELF-ORGANIZING MIXTURE MARKOV MODELS , 2010 .

[6]  Jukka Heikkonen,et al.  Context Learning with the Self Organizing , 1997 .

[7]  Pierre Baldi,et al.  Smooth On-Line Learning Algorithms for Hidden Markov Models , 1994, Neural Computation.

[8]  Marc F. J. Drossaers,et al.  An Extended Kohonen Feature Map for Sentence Recognition , 1993 .

[9]  Mikko Kurimo,et al.  Training mixture density HMMs with SOM and LVQ , 1997, Comput. Speech Lang..

[10]  D. Haussler,et al.  Hidden Markov models in computational biology. Applications to protein modeling. , 1993, Journal of molecular biology.

[11]  D. Mount.  Bioinformatics: Sequence and Genome Analysis , 2001 .

[12]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[13]  Wolfgang Rosenstiel,et al.  Automatic Cluster Detection in Kohonen's SOM , 2008, IEEE Transactions on Neural Networks.

[14]  Gregory R. Grant,et al.  Bioinformatics - The Machine Learning Approach , 2000, Comput. Chem..

[15]  Mustapha Lebbah,et al.  BeSOM : Bernoulli on Self-Organizing Map , 2007, 2007 International Joint Conference on Neural Networks.

[16]  T. Heskes.  Energy functions for self-organizing maps , 1999 .

[17]  K. Torkkola,et al.  Training continuous density hidden Markov models in association with self-organizing maps and LVQ , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[18]  Alessio Micheli,et al.  A general framework for unsupervised processing of structured data , 2004, Neurocomputing.

[19]  Andreas Stafylopatis,et al.  A Hybrid Self-Organizing Model for Sequence Analysis , 2008, 2008 20th IEEE International Conference on Tools with Artificial Intelligence.

[20]  Ben J. A. Kröse,et al.  Self-organizing mixture models , 2005, Neurocomputing.

[21]  Pierre Baldi,et al.  Hybrid Modeling, HMM/NN Architectures, and Protein Applications , 1996, Neural Computation.

[22]  Younès Bennani,et al.  The structure of verbal sequences analyzed with unsupervised learning techniques , 2007, ArXiv.

[23]  Erzsébet Merényi,et al.  Exploiting Data Topology in Visualization and Clustering of Self-Organizing Maps , 2009, IEEE Transactions on Neural Networks.

[24]  Naoyuki Tsuruta,et al.  Self-Organizing Feature Maps for HMM Based Lip-Reading , 2003, KES.

[25]  Anil K. Jain,et al.  A nonlinear projection method based on Kohonen's topology preserving maps , 1992, IEEE Trans. Neural Networks.

[26]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[27]  B. Hammer,et al.  Topographic Processing of Relational Data , 2007 .

[28]  Jukka Heikkonen,et al.  Time Series Predicition using Recurrent SOM with Local Linear Models , 1997 .

[29]  Mikko Kurimo,et al.  Using the self-organizing map to speed up the probability density estimation for speech recognition with mixture density HMMs , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[30]  Ah Chung Tsoi,et al.  Contextual Processing of Graphs using Self-Organizing Maps , 2005, ESANN.

[31]  Igor Farkas,et al.  Experimental comparison of recursive self-organizing maps for processing tree-structured data , 2010, Neurocomputing.

[32]  Tom Heskes,et al.  Transition times in self-organizing maps , 1996, Biological Cybernetics.

[33]  Lakhmi C. Jain,et al.  Self-Organizing neural networks: recent advances and applications , 2001 .

[34]  Ah Chung Tsoi,et al.  A self-organizing map for adaptive processing of structured data , 2003, IEEE Trans. Neural Networks.

[35]  Jing Kang,et al.  Prediction of Chatter in Machining Process Based on Hybrid SOM-DHMM Architecture , 2009, ICIC.

[36]  Panu Somervuo,et al.  How to make large self-organizing maps for nonvectorial data , 2002, Neural Networks.

[37]  Thomas G. Dietterich,et al.  Bioinformatics The Machine Learning Approach 2nd ed. , 2001 .

[38]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[39]  Fabrice Rossi,et al.  Fast Algorithm and Implementation of Dissimilarity Self-Organizing Maps , 2006, Neural Networks.

[40]  Alessandro Sperduti.  Neural Networks for Adaptive Processing of Structured Data , 2001, ICANN.

[41]  A. Ultsch.  Maps for the Visualization of high-dimensional Data Spaces , 2003 .

[42]  Aluizio F. R. Araújo,et al.  A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case , 2003, Neural Computation.

[43]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[44]  Barbara Hammer,et al.  Self-organizing context learning , 2004, ESANN.

[45]  David W. Mount,et al.  Bioinformatics - sequence and genome analysis (2. ed.) , 2004 .

[46]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[47]  Kurt Hornik,et al.  Convergence of learning algorithms with constant learning rates , 1991, IEEE Trans. Neural Networks.

[48]  Andreas Stafylopatis,et al.  Scaled On-line Unsupervised Learning Algorithm for a SOM-HMM Hybrid , 2011, ISCIS.

[49]  John G. Taylor,et al.  The temporal Kohönen map , 1993, Neural Networks.

[50]  Tom Heskes,et al.  Self-organizing maps, vector quantization, and mixture modeling , 2001, IEEE Trans. Neural Networks.

[51]  Ke-Lin Du,et al.  Clustering: A neural network approach , 2010, Neural Networks.

[52]  Barbara Hammer,et al.  Relational Neural Gas , 2007, KI.

[53]  Klaus Obermayer,et al.  Self-organizing maps: Generalizations and new optimization techniques , 1998, Neurocomputing.

[54]  Lakhmi C. Jain,et al.  Self-Organizing Neural Networks , 2002 .

[55]  Panu Somervuo,et al.  Self-organizing maps of symbol strings , 1998, Neurocomputing.

[56]  Valeria De Fonzo,et al.  Hidden Markov Models in Bioinformatics , 2007 .

[57]  Barbara Hammer,et al.  Topographic Mapping of Large Dissimilarity Data Sets , 2010, Neural Computation.

[58]  Pierre Baldi,et al.  Gradient descent learning algorithm overview: a general dynamical systems perspective , 1995, IEEE Trans. Neural Networks.

[59]  Panu Somervuo.  Online algorithm for the self-organizing map of symbol strings , 2004, Neural Networks.

[60]  Risto Miikkulainen,et al.  SARDNET: A Self-Organizing Feature Map for Sequences , 1994, NIPS.

[61]  Panu Somervuo.  Competing hidden Markov models on the self-organizing map , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[62]  Kadim Tasdemir.  Graph Based Representations of Density Distribution and Distances for Self-Organizing Maps , 2010, IEEE Transactions on Neural Networks.