A biologically inspired model mimicking the memory and two distinct pathways of face perception

In this paper, we propose a face perception model to mimic the biological mechanism of face perception and memory in human brain. We are mainly inspired by the fact that there are two functionally and neurologically distinct pathways after the early face perception and they both interact to process the changeable features of faces. Accordingly, our model consists of three perception parts, facial structure perception, facial expression perception and facial identity perception, which are all component-based. The structure perception has a feed-forward projection to the expression and identity perception, while the expression affects the identity perception with a modulation process. We embody the three parts referring to three bio-inspired computational models. For the facial structure perception, we utilize a cascaded-CNN (convolutional neural networks) approach to estimate the center locations of key facial components. For the facial expression perception, we propose a novel approach which exploits convolutional deep belief networks (CDBN) to spontaneously locate the places containing the most discriminative information and synchronously complete the feature learning and feature selection. For the facial identity perception, we propose an approach which adopts the hierarchical max-pooling (HMAX) model to encode notable characteristics of facial components and utilizes a new memory formation integrating the preliminary decision, expression modulation and final decision process. We evaluate our model through a series of experiments and the experimental results demonstrate its rationality and effectiveness.

[1]  John M. Gardiner,et al.  An appreciation of generate-recognize theory of recall , 1979 .

[2]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[3]  Fred Dretske What We See , 2010 .

[4]  Laurent Itti,et al.  Applying computational tools to predict gaze direction in interactive visual environments , 2008, TAP.

[5]  Geoffrey E. Hinton,et al.  Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes , 2007, NIPS.

[6]  H. Ellis,et al.  Identification of Familiar and Unfamiliar Faces from Internal and External Features: Some Implications for Theories of Face Recognition , 1979, Perception.

[7]  A. Young,et al.  Matching familiar and unfamiliar faces on identity and expression , 1986, Psychological research.

[8]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[9]  Zhengyou Zhang,et al.  Comparison between geometry-based and Gabor-wavelets-based facial expression recognition using multi-layer perceptron , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[10]  Alex Martin,et al.  Semantic memory and the brain: structure and processes , 2001, Current Opinion in Neurobiology.

[11]  Tomaso Poggio,et al.  'United' we stand. , 2002 .

[12]  PietikainenMatti,et al.  Face Description with Local Binary Patterns , 2006 .

[13]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Bhuvana Ramabhadran,et al.  Deep belief nets for natural language call-routing , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[15]  Hong Qiao,et al.  Introducing Memory and Association Mechanism Into a Biologically Inspired Visual Model , 2014, IEEE Transactions on Cybernetics.

[16]  D. Hubel,et al.  Receptive fields of single neurones in the cat's striate cortex , 1959, The Journal of physiology.

[17]  Bo Zhang,et al.  Enhanced HMAX model with feedforward feature learning for multiclass categorization , 2015, Front. Comput. Neurosci..

[18]  T. Poggio,et al.  A model of V4 shape selectivity and invariance. , 2007, Journal of neurophysiology.

[19]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[20]  Honglak Lee,et al.  Unsupervised feature learning for audio classification using convolutional deep belief networks , 2009, NIPS.

[21]  Nicholas B. Turk-Browne,et al.  Face-Specific Resting Functional Connectivity between the Fusiform Gyrus and Posterior Superior Temporal Sulcus , 2010, Front. Hum. Neurosci..

[22]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[23]  Geoffrey E. Hinton,et al.  3D Object Recognition with Deep Belief Nets , 2009, NIPS.

[24]  Honglak Lee,et al.  Learning hierarchical representations for face verification with convolutional deep belief networks , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  A. Young,et al.  Understanding the recognition of facial identity and facial expression , 2005, Nature Reviews Neuroscience.

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[28]  Honglak Lee,et al.  Unsupervised learning of hierarchical representations with convolutional deep belief networks , 2011, Commun. ACM.

[29]  Xiaogang Wang,et al.  Deep Convolutional Network Cascade for Facial Point Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Christof Koch,et al.  Attentional Selection for Object Recognition - A Gentle Way , 2002, Biologically Motivated Computer Vision.

[31]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[32]  Hong Qiao,et al.  An improved eLBPH method for facial identity recognition: Expression-specific weighted local binary pattern histogram , 2015, 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[33]  A. Mack,et al.  What we see: Inattention and the capture of attention by meaning , 2002, Consciousness and Cognition.

[34]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[35]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[36]  Wenfeng Feng,et al.  Three stages of facial expression processing: ERP study with rapid serial visual presentation , 2010, NeuroImage.

[37]  J.J. Hopfield,et al.  Artificial neural networks , 1988, IEEE Circuits and Devices Magazine.

[38]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[39]  A. Young,et al.  Understanding face recognition. , 1986, British journal of psychology.

[40]  David I. Perrett,et al.  Neurophysiology of shape processing , 1993, Image Vis. Comput..

[41]  D I Perrett,et al.  Organization and functions of cells responsive to faces in the temporal cortex. , 1992, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[42]  Richard J. Harris,et al.  Neural Responses to Expression and Gaze in the Posterior Superior Temporal Sulcus Interact with Facial Identity , 2012, Cerebral cortex.

[43]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[44]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[45]  Giovanni Maria Carlomagno,et al.  Heat flux sensors and infrared thermography , 2007, J. Vis..

[46]  Leslie G. Ungerleider,et al.  Discrete Cortical Regions Associated with Knowledge of Color and Knowledge of Action , 1995, Science.

[47]  Xuelong Li,et al.  Enhanced Biologically Inspired Model for Object Recognition , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[48]  Hiroyuki Oya,et al.  Decoding Face Information in Time, Frequency and Space from Direct Intracranial Recordings of the Human Brain , 2008, PloS one.

[49]  Edgar Bermudez Contreras,et al.  Attention can improve a simple model for object recognition , 2008, Image Vis. Comput..

[50]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  V Bruce,et al.  Familiarisation with Faces Selectively Enhances Sensitivity to Changes Made to the Eyes , 2001, Perception.

[52]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[53]  E. Fox,et al.  Anxiety modulates the degree of attentive resources required to process emotional faces , 2005, Cognitive, affective & behavioral neuroscience.

[54]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[55]  I. Fraser,et al.  Reaction Time Measures of Feature Saliency in Schematic Faces , 1990, Perception.

[56]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[57]  T. Poggio,et al.  I think I know that face... , 1996, Nature.

[58]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[59]  Andrew D. Engell,et al.  Distributed representations of dynamic facial expressions in the superior temporal sulcus. , 2010, Journal of vision.

[60]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  David G. Lowe,et al.  Multiclass Object Recognition with Sparse, Localized Features , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[62]  J. Shepherd Studies of cue saliency , 1981 .

[63]  R. Dolan,et al.  fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. , 2004, Journal of neurophysiology.

[64]  Jean Meunier,et al.  Emotion recognition using dynamic grid-based HoG features , 2011, Face and Gesture 2011.

[65]  Ping Liu,et al.  Facial Expression Recognition via a Boosted Deep Belief Network , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[66]  Fengfu Li,et al.  Biologically Inspired Visual Model With Preliminary Cognition and Active Attention Adjustment , 2015, IEEE Transactions on Cybernetics.

[67]  Thomas Wennekers,et al.  The role of feedback in a hierarchical model of object perception. , 2011, Advances in experimental medicine and biology.