Artificial Vision by Deep CNN Neocognitron

Deep convolutional neural networks (deep CNNs) show a large power for robust recognition of visual patterns. The neocognitron, which was first proposed by Fukushima (1979), is recognized as the origin of deep CNNs. Its architecture was suggested by the neurophysiological findings on the visual systems of mammals. It acquires the ability to recognize visual patterns robustly through learning. Although the neocognitron has a long history, improvements of the network are still continuing. This article discusses the recent neocognitron, focusing on differences from the conventional deep CNNs. Some other functions of the visual system can also be realized by networks extended from the neocognitron, for example, recognition of partly occluded patterns, the mechanism of selective attention, and so on.

[1]  Taghi M. Khoshgoftaar,et al.  A survey on Image Data Augmentation for Deep Learning , 2019, Journal of Big Data.

[2]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[3]  Kunihiko Fukushima,et al.  Recent advances in the deep CNN neocognitron , 2019, Nonlinear Theory and Its Applications, IEICE.

[4]  Masayuki Kikuchi,et al.  Neural Network Model of the Visual System: Binding Form and Motion , 1996, Neural Networks.

[5]  Stephen Grossberg,et al.  Recognition and segmentation of connected characters with selective attention , 1994, Neural Networks.

[6]  Hideki Hashimoto,et al.  Recognition and Segmentation of Components of a Face by a Multi-Resolution Neural Network , 1997, ICANN.

[7]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[8]  Kunihiko Fukushima,et al.  Restoring partly occluded patterns: a neural network model , 2005, Neural Networks.

[9]  K. Fukushima.  Neural network model for selective attention in visual pattern recognition and associative recall. , 1987, Applied optics.

[10]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[11]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[12]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[13]  Kunihiko Fukushima.  Efficient IntVec: High recognition rate with reduced computational cost , 2019, Neural Networks.

[14]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[15]  S LewMichael,et al.  Deep learning for visual understanding , 2016 .

[16]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[17]  Takayuki Ito,et al.  Neocognitron: A neural network model for a mechanism of visual pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[18]  Hayaru Shouno,et al.  Deep convolutional network neocognitron: Improved Interpolating-Vector , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[19]  Kunihiko Fukushima,et al.  Training multi-layered neural network neocognitron , 2013, Neural Networks.

[20]  Kunihiko Fukushima,et al.  Interpolating vectors for robust pattern recognition , 2007, Neural Networks.

[21]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[22]  Masashi Tanigawa,et al.  Use of different thresholds in learning and recognition , 1996, Neurocomputing.

[23]  Kunihiko Fukushima,et al.  Artificial vision by multi-layered neural networks: Neocognitron and its advances , 2013, Neural Networks.

[24]  Kunihiko Fukushima.  Margined winner-take-all: New learning rule for pattern recognition , 2018, Neural Networks.

[25]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[26]  Kunihiko Fukushima.  One-Shot Learning with Feedback for Multi-layered Convolutional Network , 2014, ICANN.

[27]  Kunihiko Fukushima.  Recognition of partly occluded patterns: a neural network model , 2001, Biological Cybernetics.

[28]  Kunihiko Fukushima,et al.  Increasing robustness against background noise: Visual pattern recognition by a neocognitron , 2011, Neural Networks.