A Spiking Neural Network Based Cortex-Like Mechanism and Application to Facial Expression Recognition

In this paper, we present a quantitative, highly structured cortex-simulated model, which can be simply described as feedforward, hierarchical simulation of ventral stream of visual cortex using biologically plausible, computationally convenient spiking neural network system. The motivation comes directly from recent pioneering works on detailed functional decomposition analysis of the feedforward pathway of the ventral stream of visual cortex and developments on artificial spiking neural networks (SNNs). By combining the logical structure of the cortical hierarchy and computing power of the spiking neuron model, a practical framework has been presented. As a proof of principle, we demonstrate our system on several facial expression recognition tasks. The proposed cortical-like feedforward hierarchy framework has the merit of capability of dealing with complicated pattern recognition problems, suggesting that, by combining the cognitive models with modern neurocomputational approaches, the neurosystematic approach to the study of cortex-like mechanism has the potential to extend our knowledge of brain mechanisms underlying the cognitive analysis and to advance theoretical models of how we recognize face or, more specifically, perceive other people's facial expression in a rich, dynamic, and complex environment, providing a new starting point for improved models of visual cortex-like mechanism.

[1]  Zia-ur Rahman,et al.  Multi-scale retinex for color image enhancement , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[2]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[3]  Michel Vidal-Naquet,et al.  Visual features of intermediate complexity and their use in classification , 2002, Nature Neuroscience.

[4]  Wendy L. Braje,et al.  Illumination effects in face recognition , 1998, Psychobiology.

[5]  Doris Y. Tsao,et al.  A face feature space in the macaque temporal lobe , 2009, Nature Neuroscience.

[6]  Michael J. Lyons,et al.  Evidence and a computational explanation of cultural differences in facial expression recognition. , 2010, Emotion.

[7]  Takeo Kanade,et al.  Evaluation of Gabor-wavelet-based facial action unit recognition in image sequences of increasing complexity , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[8]  Rachael E. Jack,et al.  Internal representations reveal cultural diversity in expectations of facial expressions of emotion. , 2012, Journal of experimental psychology. General.

[9]  Thomas Serre,et al.  A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex , 2005 .

[10]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[11]  Rachael E. Jack,et al.  Cultural Confusions Show that Facial Expressions Are Not Universal , 2009, Current Biology.

[12]  R. Dolan,et al.  Emotion, Cognition, and Behavior , 2002, Science.

[13]  Tim Gollisch,et al.  Modeling Single-Neuron Dynamics and Computations: A Balance of Detail and Abstraction , 2006, Science.

[14]  S. Thorpe,et al.  Surfing a spike wave down the ventral stream , 2002, Vision Research.

[15]  Aapo Hyvärinen,et al.  Topographic Independent Component Analysis , 2001, Neural Computation.

[16]  Terrence J Sejnowski,et al.  Communication in Neuronal Networks , 2003, Science.

[17]  Kenneth D. Miller,et al.  Adaptive filtering enhances information transmission in visual cortex , 2006, Nature.

[18]  Simei Gomes Wysoski,et al.  Evolving spiking neural networks for audiovisual information processing , 2010, Neural Networks.

[19]  Simei Gomes Wysoski,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008, Neurocomputing.

[20]  C. E. Connor,et al.  A New Viewpoint on Faces , 2010, Science.

[21]  Jacques Gautrais,et al.  SpikeNET: A simulator for modeling large networks of integrate and fire neurons , 1999, Neurocomputing.

[22]  Zeng-Guang Hou,et al.  Multiple kernel learning with ICA: Local discriminative image descriptors for recognition , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[23]  Thomas Serre,et al.  Models of visual cortex , 2013, Scholarpedia.

[24]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[25]  Doris Y. Tsao,et al.  Functional Compartmentalization and Viewpoint Generalization Within the Macaque Face-Processing System , 2010, Science.

[26]  J. A. Aznar-Casanova,et al.  Patterns of brain asymmetry in the perception of positive and negative facial expressions , 2009, Laterality.

[27]  Li Zhaoping,et al.  Theoretical understanding of the early visual processes by data compression and data selection , 2006, Network.

[28]  Patrick Shen-Pei Wang,et al.  Performance Comparisons of Facial Expression Recognition in Jaffe Database , 2008, Int. J. Pattern Recognit. Artif. Intell..

[29]  Nikola K. Kasabov,et al.  To spike or not to spike: A probabilistic spiking neuron model , 2010, Neural Networks.

[30]  Zeng-Guang Hou,et al.  Spiking neural networks based cortex like mechanism: A case study for facial expression recognition , 2011, The 2011 International Joint Conference on Neural Networks.

[31]  Heiko Wersing,et al.  Learning Optimized Features for Hierarchical Models of Invariant Object Recognition , 2003, Neural Computation.

[32]  W. Zheng,et al.  Facial expression recognition using kernel canonical correlation analysis (KCCA) , 2006, IEEE Transactions on Neural Networks.

[33]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[34]  Fei Cheng,et al.  Facial Expression Recognition in JAFFE Dataset Based on Gaussian Process Classification , 2010, IEEE Transactions on Neural Networks.

[35]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[36]  Wulfram Gerstner,et al.  Spiking Neuron Models: An Introduction , 2002 .

[37]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[38]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  J L Gallant,et al.  Sparse coding and decorrelation in primary visual cortex during natural vision. , 2000, Science.

[40]  G. Cottrell,et al.  EMPATH: A Neural Network that Categorizes Facial Expressions , 2002, Journal of Cognitive Neuroscience.

[41]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[42]  Lianwen Jin,et al.  A New Facial Expression Recognition Method Based on Local Gabor Filter Bank and PCA plus LDA , 2006 .

[43]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[44]  Shimon Ullman,et al.  Computation of pattern invariance in brain-like structures , 1999, Neural Networks.

[45]  Tomaso Poggio,et al.  Learning a dictionary of shape-components in visual cortex: comparison with neurons, humans and machines , 2006 .

[46]  C. Gross Brain, Vision, Memory: Tales in the History of Neuroscience , 1998 .

[47]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Eric T. Carlson,et al.  A neural code for three-dimensional object shape in macaque inferotemporal cortex , 2008, Nature Neuroscience.

[49]  Kunihiko Fukushima,et al.  Neocognitron: A Self-Organizing Neural Network Model for a Mechanism of Visual Pattern Recognition , 1982 .

[50]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[51]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .