Deep Learning Self-Organizing Map of Convolutional Layers

The Self-Organizing Convolutional Map (SOCOM) combines convolutional neural networks, clustering via self-organizing maps, and learning through gradient backpropagation into a novel unified unsupervised deep architecture. The proposed clustering and training procedures reflect the model’s degree of integration and synergy between its constituting modules. The SOCOM prototype is in position to carry out unsupervised classification and clustering tasks based upon the distributed higher level representations that are produced by its underlying convolutional deep architecture, without necessitating target or label information at any stage of its training and inference operations. Due to its convolutional component SOCOM has the intrinsic capability to model signals consisting of one or more channels like grayscale and colored images.

[1]  Yannis Papanikolaou,et al.  Denoising Autoencoder Self-Organizing Map (DASOM) , 2018, Neural Networks.

[2]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Purang Abolmaesumi,et al.  Deep Neural Maps , 2018, ICLR.

[4]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Chathurika S. Wickramasinghe,et al.  Deep Self-Organizing Maps for Unsupervised Image Classification , 2019, IEEE Transactions on Industrial Informatics.

[7]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[9]  David Friedlander,et al.  Pattern Analysis with Layered Self-Organizing Maps , 2018, ArXiv.

[10]  Pedro H. M. Braga,et al.  Deep Categorization with Semi-Supervised Self-Organizing Maps , 2020, 2020 International Joint Conference on Neural Networks (IJCNN).

[11]  Oliver Lemon.  Incremental On-Line Learning of Object Classes using a Combination of Self-Organizing Incremental Neural Networks and Deep Convolutional Neural Networks* , 2016 .

[12]  Daniel L. Marino,et al.  Deep Self-Organizing Maps for Visual Data Mining , 2018, 2018 11th International Conference on Human System Interaction (HSI).

[13]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[14]  Alex Krizhevsky,et al.  One weird trick for parallelizing convolutional neural networks , 2014, ArXiv.

[15]  Sultan Almotairi,et al.  Deep Convolutional Self-Organizing Map Network for Robust Handwritten Digit Recognition , 2020, IEEE Access.

[16]  Jurgen Brauer,et al.  CSNNs: Unsupervised, Backpropagation-Free Convolutional Neural Networks for Representation Learning , 2019, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA).

[17]  Andreas Stafylopatis,et al.  Self-Organizing Hidden Markov Model Map (SOHMMM) , 2013, Neural Networks.

[18]  Yihong Gong,et al.  Deep Self-Organizing Map for visual classification , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[19]  Christos Ferles,et al.  Self-Organizing Hidden Markov Model Map (SOHMMM): Biological Sequence Clustering and Cluster Visualization. , 2017, Methods in molecular biology.

[20]  Qi Tian,et al.  Deep Supervised Quantization by Self-Organizing Map , 2017, ACM Multimedia.

[21]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[22]  T. Heskes.  Energy functions for self-organizing maps , 1999 .

[23]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[24]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.