Pattern Recognition by Hierarchical Temporal Memory

Hierarchical Temporal Memory (HTM) is still largely unknown by the pattern recognition community and only a few studies have been published in the scientific literature. This paper reviews HTM architecture and related learning algorithms by using formal notation and pseudocode description. Novel approaches are then proposed to encode coincidence-group membership (fuzzy grouping) and to derive temporal groups (maxstab temporal clustering). Systematic experiments on three line-drawing datasets have been carried out to better understand HTM peculiarities and to extensively compare it against other well-know pattern recognition approaches. Our results prove the effectiveness of the new algorithms introduced and that HTM, even if still in its infancy, compares favorably with other existing technologies.

[1]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[2]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[3]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[5]  Yiannis Aloimonos,et al.  Active vision , 2004, International Journal of Computer Vision.

[6]  E. Sackinger,et al.  Neural-Network and k-Nearest-neighbor Classifiers , 1991 .

[7]  Yann LeCun,et al.  Efficient Pattern Recognition Using a New Transformation Distance , 1992, NIPS.

[8]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[9]  Bernhard Schölkopf,et al.  Incorporating Invariances in Support Vector Learning Machines , 1996, ICANN.

[10]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[13]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[14]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[15]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[16]  Terrence J. Sejnowski,et al.  Slow Feature Analysis: Unsupervised Learning of Invariances , 2002, Neural Computation.

[17]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[18]  Eric Horvitz,et al.  Layered representations for learning and inferring office activity from multiple sensory channels , 2004, Comput. Vis. Image Underst..

[19]  J. Hawkins,et al.  On Intelligence , 2004 .

[20]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[21]  C. Koch.  The quest for consciousness : a neurobiological approach , 2004 .

[22]  Yoram Singer,et al.  The Hierarchical Hidden Markov Model: Analysis and Applications , 1998, Machine Learning.

[23]  D. George,et al.  A hierarchical Bayesian model of invariant pattern recognition in the visual cortex , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[24]  Hairong Lv,et al.  Handwritten Digit Recognition Using Low Rank Approximation Based Competitive Neural Network , 2006, ISNN.

[25]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[26]  John Thornton,et al.  Robust Character Recognition Using a Hierarchical Bayesian Network , 2006, Australian Conference on Artificial Intelligence.

[27]  Cordelia Schmid,et al.  Dataset Issues in Object Recognition , 2006, Toward Category-Level Object Recognition.

[28]  Saulius Juozas Garalevicius.  Memory-Prediction Framework for Pattern Recognition: Performance and Suitability of the Bayesian Model of Visual Cortex , 2007, FLAIRS Conference.

[29]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Bruce A. Bobier.  Handwritten Digit Recognition using Hierarchical Temporal Memory , 2007 .

[31]  Marc'Aurelio Ranzato,et al.  Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[33]  P. Baranyi,et al.  Object Categorization Using VFA-generated Nodemaps and Hierarchical Temporal Memories , 2007, 2007 IEEE International Conference on Computational Cybernetics.

[34]  Bruce A. Bobier,et al.  Content-based image retrieval using hierarchical temporal memory , 2008, ACM Multimedia.

[35]  Dileep George,et al.  How the brain might work: a hierarchical and temporal model for learning and recognition , 2008 .

[36]  Lei Wang,et al.  Object Recognition Using a Bayesian Network Imitating Human Neocortex , 2009, 2009 2nd International Congress on Image and Signal Processing.

[37]  Yann LeCun,et al.  EBLearn: Open-Source Energy-Based Learning in C++ , 2009, 2009 21st IEEE International Conference on Tools with Artificial Intelligence.

[38]  Lorenzo Rosasco,et al.  On Invariance in Hierarchical Models , 2009, NIPS.

[39]  Dileep George,et al.  Towards a Mathematical Theory of Cortical Micro-circuits , 2009, PLoS Comput. Biol..

[40]  Derek C. Rose,et al.  Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.