Self-organizing neurons: toward brain-inspired unsupervised learning

During the last years, Deep Neural Networks have reached the highest performances in image classification. Nevertheless, such a success is mostly based on supervised and off-line learning: they require thus huge labeled datasets for learning, and once it is done, they cannot adapt to any change in the data from the environment. In the context of brain-inspired computing, we apply Kohonen-based Self-Organizing Maps for unsupervised learning without labels, and we explore original extensions such as the Dynamic SOM that enables continuous learning and the Pruning Cellular SOM that includes synaptic pruning in neuromorphic circuits. After presenting the three models and the experimental setup for MNIST classification, we compare different methods for automatic labeling based on very few labeled data (1% of the training dataset), and then we compare the performances of the three Kohonen-based SelfOrganizing Maps with STDP-based Spiking Neural Networks in terms of accuracy, dynamicity and scalability.

[1]  Yann Boniface,et al.  Dynamic self-organising map , 2011, Neurocomputing.

[2]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[3]  Laurent Rodriguez,et al.  A distributed cellular approach of large scale SOM models for hardware implementation , 2018, 2018 IEEE International Conference on Image Processing, Applications and Systems (IPAS).

[4]  Bernard Girau,et al.  Neuromorphic hardware as a self-organizing computing system , 2018, ArXiv.

[5]  Konrad P. Körding,et al.  Toward an Integration of Deep Learning and Neuroscience , 2016, bioRxiv.

[6]  Yihong Gong,et al.  Deep Self-Organizing Map for visual classification , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[7]  Milos Manic,et al.  Parallalizable deep self-organizing maps for image classification , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[8]  Yoshua Bengio,et al.  STDP as presynaptic activity times rate of change of postsynaptic activity , 2015, 1509.05936.

[9]  Benoît Miramond,et al.  Confronting machine-learning with neuroscience for neuromorphic architectures design , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[10]  Hananel Hazan,et al.  Unsupervised Learning with Self-Organizing Spiking Neural Networks , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[11]  Irem Dikmen,et al.  Comparing the performance of traditional cluster analysis, self-organizing maps and fuzzy C-means method for strategic grouping , 2009, Expert Syst. Appl..

[12]  Daniel J. Saunders,et al.  STDP Learning of Image Patches with Convolutional Spiking Neural Networks , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[13]  L. Buydens,et al.  Supervised Kohonen networks for classification problems , 2006 .

[14]  Kingson Man,et al.  Convergent and invariant object representations for sight, sound, and touch , 2015, Human brain mapping.

[15]  Catherine D. Schuman,et al.  A Survey of Neuromorphic Computing and Neural Networks in Hardware , 2017, ArXiv.

[16]  Erkki Oja,et al.  Engineering applications of the self-organizing map , 1996, Proc. IEEE.

[17]  Bertrand Granado,et al.  Toward a Sparse Self-Organizing Map for Neuromorphic Architectures , 2015, ACM J. Emerg. Technol. Comput. Syst..

[18]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[19]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[20]  Laurent Rodriguez,et al.  A neural model for hardware plasticity in artificial vision systems , 2013, 2013 Conference on Design and Architectures for Signal and Image Processing.