Region-specific fMRI dictionary for decoding face verification in humans

This paper focuses on decoding the process of face verification in the human brain using fMRI responses. 2400 fMRI responses are collected from different participants while they perform face verification on genuine and imposter stimuli face pairs. The first part of the paper analyzes the responses covering both cognitive and fMRI neuro-imaging results. With an average verification accuracy of 64.79% by human participants, the results of the cognitive analysis depict that the performance of female participants is significantly higher than the male participants with respect to imposter pairs. The results of the neuro-imaging analysis identifies regions of the brain such as the left fusiform gyrus, caudate nucleus, and superior frontal gyrus that are activated when participants perform face verification tasks. The second part of the paper proposes a novel two-level fMRI dictionary learning approach to predict if the stimuli observed is genuine or imposter using the brain activation data for selected regions. A comparative analysis with existing machine learning techniques illustrates that the proposed approach yields at least 4.5% higher classification accuracy than other algorithms. It is envisioned that the result of this study is the first step in designing brain-inspired automatic face verification algorithms.

[1]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Istvan Molnar-Szakacs,et al.  Self-face recognition activates a frontoparietal “mirror” network in the right hemisphere: an event-related fMRI study , 2005, NeuroImage.

[3]  R. Dolan,et al.  Contrast polarity and face recognition in the human fusiform gyrus , 1999, Nature Neuroscience.

[4]  Dinggang Shen,et al.  Classifying spatial patterns of brain activity with machine learning methods: Application to lie detection , 2005, NeuroImage.

[5]  Kenneth F. Valyear,et al.  The fusiform face area is not sufficient for face recognition: Evidence from a patient with dense prosopagnosia and no occipital face area , 2006, Neuropsychologia.

[6]  Mary L. Phillips,et al.  What Does Brain Response to Neutral Faces Tell Us about Major Depression? Evidence from Machine Learning and fMRI , 2013, PloS one.

[7]  Vaidehi S. Natu,et al.  The neural processing of familiar and unfamiliar faces: a review and synopsis. , 2011, British journal of psychology.

[8]  Sungho Tak,et al.  A Data-Driven Sparse GLM for fMRI Analysis Using Sparse Dictionary Learning With MDL Criterion , 2011, IEEE Transactions on Medical Imaging.

[9]  S. Platek,et al.  Is family special to the brain? An event-related fMRI study of familiar, familial, and self-face recognition , 2009, Neuropsychologia.

[10]  C. Frith,et al.  Modulation of human visual cortex by crossmodal spatial attention. , 2000, Science.

[11]  Himanshu S. Bhatt,et al.  Recognizing Surgically Altered Face Images Using Multiobjective Evolutionary Algorithm , 2013, IEEE Transactions on Information Forensics and Security.

[12]  D. Keeble,et al.  The Significance of Hair for Face Recognition , 2012, PloS one.

[13]  M. Tarr,et al.  The Fusiform Face Area is Part of a Network that Processes Faces at the Individual Level , 2000, Journal of Cognitive Neuroscience.

[14]  Alice J. O'Toole,et al.  Dissociable Neural Patterns of Facial Identity across Changes in Viewpoint , 2010, Journal of Cognitive Neuroscience.

[15]  Lei Zhang,et al.  Projective dictionary pair learning for pattern classification , 2014, NIPS.

[16]  Daksha Yadav,et al.  Multiple Projective Dictionary Learning to Detect Plastic Surgery for Face Verification , 2015, IEEE Access.

[17]  Richa Singh,et al.  Face recognition with disguise and single gallery images , 2009, Image Vis. Comput..

[18]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[19]  Bruno Rossion,et al.  Hemispheric Asymmetries for Whole-Based and Part-Based Face Processing in the Human Fusiform Gyrus , 2000, Journal of Cognitive Neuroscience.

[20]  N. Kanwisher,et al.  The Cognitive and Neural Development of Face Recognition in Humans , 2009 .

[21]  I Daubechies,et al.  Independent component analysis for brain fMRI does not select for independence , 2009 .

[22]  Joy Hirsch,et al.  Functional Specialization within the Medial Frontal Gyrus for Perceptual Go/No-Go Decisions Based on What, When, and Where Related Information: An fMRI Study , 2005, Journal of Cognitive Neuroscience.

[23]  Norbert Krüger,et al.  Face Recognition by Elastic Bunch Graph Matching , 1997, CAIP.

[24]  Ingrid R. Olson,et al.  Beyond the FFA: The role of the ventral anterior temporal lobes in face processing , 2014, Neuropsychologia.

[25]  M. Just,et al.  Identifying Emotions on the Basis of Neural Activation , 2013, PloS one.

[26]  B. Dubois,et al.  Functions of the left superior frontal gyrus in humans: a lesion study. , 2006, Brain : a journal of neurology.

[27]  T. Allison,et al.  Face-sensitive regions in human extrastriate cortex studied by functional MRI. , 1995, Journal of neurophysiology.

[28]  A. Majumdar,et al.  Hierarchical Representation Learning for Kinship Verification , 2017, IEEE Transactions on Image Processing.

[29]  John Ashburner,et al.  Kernel methods for fMRI pattern prediction , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[30]  Hananel Hazan,et al.  Classification from generation: Recognizing deep grammatical information during reading from rapid event-related fMRI , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[31]  E. Ngan,et al.  Functional brain organization of preparatory attentional control in visual search , 2013, Brain Research.

[32]  Peter Fransson,et al.  Access to the Published Version May Require Journal Subscription. Published with Permission From: Elsevier Fmri Activity in the Medial Temporal Lobe during Famous Face Processing , 2022 .

[33]  Richa Singh,et al.  MDLFace: Memorability augmented deep learning for video face recognition , 2014, IEEE International Joint Conference on Biometrics.

[34]  E. Tulving,et al.  Episodic and declarative memory: Role of the hippocampus , 1998, Hippocampus.

[35]  T. Allison,et al.  Face-Specific Processing in the Human Fusiform Gyrus , 1997, Journal of Cognitive Neuroscience.

[36]  J. Haxby,et al.  Neural response to the visual familiarity of faces , 2006, Brain Research Bulletin.

[37]  D. Pitcher,et al.  The role of the occipital face area in the cortical face perception network , 2011, Experimental Brain Research.

[38]  Ingemar J. Cox,et al.  Feature-based face recognition using mixture-distance , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Saeid Sanei,et al.  Fast and incoherent dictionary learning algorithms with application to fMRI , 2015, Signal Image Video Process..

[40]  Jia Liu,et al.  Perception of Face Parts and Face Configurations: An fMRI Study , 2010, Journal of Cognitive Neuroscience.

[41]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[42]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[43]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Charles D. Smith,et al.  Neural substrates of facial emotion processing using fMRI. , 2001, Brain research. Cognitive brain research.

[45]  Alice J. O'Toole,et al.  Spatiotemporal changes in neural response patterns to faces varying in visual familiarity , 2015, NeuroImage.

[46]  K. Grill-Spector,et al.  Electrical Stimulation of Human Fusiform Face-Selective Regions Distorts Face Perception , 2012, The Journal of Neuroscience.

[47]  Leslie G. Ungerleider,et al.  Face encoding and recognition in the human brain. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Marco Iacoboni,et al.  The self across the senses: an fMRI study of self-face and self-voice recognition. , 2008, Social cognitive and affective neuroscience.

[49]  D. Louis Collins,et al.  Unbiased average age-appropriate atlases for pediatric studies , 2011, NeuroImage.

[50]  Christa Neuper,et al.  To retrieve or to calculate? Left angular gyrus mediates the retrieval of arithmetic facts during problem solving , 2009, Neuropsychologia.