Pattern Recognition: Invariance Learning in Convolutional Auto Encoder Network

The ability of the human visual processing system to accommodate and retain clear understanding or identification of patterns irrespective of their orientations is quite remarkable. Conversely, pattern invariance, a common problem in intelligent recognition systems is not one that can be overemphasized; obviously, one’s definition of an intelligent system broadens considering the large variability with which the same patterns can occur. This research investigates and reviews the performance of convolutional networks, and its variant, convolutional auto encoder networks when tasked with recognition problems considering invariances such as translation, rotation, and scale. While, various patterns can be used to validate this query, handwritten Yoruba vowel characters have been used in this research. Databases of images containing patterns with constraints of interest are collected, processed, and used to train and simulate the designed networks. We provide extensive architectural and learning paradigms review of the considered networks, in view of how built-in invariance is learned. Lastly, we provide a comparative analysis of achieved error rates against back propagation neural networks, denoising auto encoder, stacked denoising auto encoder, and deep belief network.

[1]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[2]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Michel Doucet,et al.  Rotation, scale, and translation invariant pattern recognition using feature extraction , 1997, Defense, Security, and Sensing.

[4]  Li Deng,et al.  An Overview of Deep-Structured Learning for Information Processing , 2011 .

[5]  Heiko Wersing,et al.  Learning Optimized Features for Hierarchical Models of Invariant Object Recognition , 2003, Neural Computation.

[6]  Nate Sauder.  Encoded Invariance in Convolutional Neural Networks , 2014 .

[7]  Sukhjeet Kaur Ranade,et al.  Empirical Analysis of Rotation Invariance in Moment Coefficients , 2015 .

[8]  Kunihiko Fukushima,et al.  Recognition and segmentation of connected characters with selective attention , 1993, Neural Networks.

[9]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[10]  Luca Maria Gambardella,et al.  Max-pooling convolutional neural networks for vision-based hand gesture recognition , 2011, 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA).

[11]  A. Khashman,et al.  Intelligent coin identification system , 2006, 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control.

[12]  Linda G. Shapiro,et al.  Computer Vision , 2001 .

[13]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[14]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[15]  Adnan Khashman,et al.  Deep Learning in Character Recognition Considering Pattern Invariance Constraints , 2015 .

[16]  Joel Z. Leibo,et al.  Learning Generic Invariances in Object Recognition: Translation and Scale , 2010 .

[17]  Rehab F. Abdel-Kader,et al.  Rotation-Invariant Pattern Recognition Approach Using Extracted Descriptive Symmetrical Patterns , 2012 .

[18]  Luca Maria Gambardella,et al.  Flexible, High Performance Convolutional Neural Networks for Image Classification , 2011, IJCAI.

[19]  Jake Bouvrie,et al.  Notes on Convolutional Neural Networks , 2006 .

[20]  C. Neubauer.  Recognition of Handwritten Digits and Human Faces by Convolutional Neural Networks , 1996 .

[21]  Guangyi Chen,et al.  Invariant Fourier-wavelet descriptor for pattern recognition , 1999, Pattern Recognit..