The Face of Image Reconstruction: Progress, Pitfalls, Prospects

[1]  M. Seghier,et al.  A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. , 2003, Brain : a journal of neurology.

[2]  Connor J. Parde,et al.  Face Space Representations in Deep Convolutional Neural Networks , 2018, Trends in Cognitive Sciences.

[3]  G B Stanley,et al.  Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus , 1999, The Journal of Neuroscience.

[4]  Philippe G. Schyns,et al.  Revealing the information contents of memory within the stimulus information representation framework , 2020, Philosophical Transactions of the Royal Society B.

[5]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[6]  D. Plaut,et al.  Face-Space Architectures , 2013, Psychological science.

[7]  Tom Heskes,et al.  Linear reconstruction of perceived images from human brain activity , 2013, NeuroImage.

[8]  Luca Ambrogioni,et al.  Generative adversarial networks for reconstructing natural images from brain activity , 2017, NeuroImage.

[9]  Carlos F. Benitez-Quiroz,et al.  Facial color is an efficient mechanism to visually transmit emotion , 2018, Proceedings of the National Academy of Sciences.

[10]  A. Young,et al.  Are We Face Experts? , 2018, Trends in Cognitive Sciences.

[11]  D. Maurer,et al.  The many faces of configural processing , 2002, Trends in Cognitive Sciences.

[12]  Guohua Shen,et al.  Deep image reconstruction from human brain activity , 2017, bioRxiv.

[13]  P. Sinha,et al.  Role of ordinal contrast relationships in face encoding , 2009, Proceedings of the National Academy of Sciences.

[14]  C. Collin,et al.  On the particular vulnerability of face recognition to aging: a review of three hypotheses , 2015, Front. Psychol..

[15]  Marlene Behrmann,et al.  Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis , 2011, Proceedings of the National Academy of Sciences.

[16]  A. O’toole Face Perception, Psychology of , 2006 .

[17]  A. O'Toole,et al.  Prototype-referenced shape encoding revealed by high-level aftereffects , 2001, Nature Neuroscience.

[18]  Dan Nemrodov,et al.  Memory and Perception-based Facial Image Reconstruction , 2017, Scientific Reports.

[19]  Rachael E. Jack,et al.  Toward a Social Psychophysics of Face Communication , 2017, Annual review of psychology.

[20]  E. Mori,et al.  Pareidolias: complex visual illusions in dementia with Lewy bodies , 2012, Brain : a journal of neurology.

[21]  J. S. Guntupalli,et al.  Disentangling the Representation of Identity from Head View Along the Human Face Processing Pathway , 2016, bioRxiv.

[22]  Connor J. Parde,et al.  Deep convolutional neural networks in the face of caricature , 2018, Nature Machine Intelligence.

[23]  R. VanRullen,et al.  Reconstructing faces from fMRI patterns using deep generative neural networks. , 2019 .

[24]  P. Schyns,et al.  Superstitious Perceptions Reveal Properties of Internal Representations , 2003, Psychological science.

[25]  S. Kochen,et al.  Single Neuron Coding of Identity in the Human Hippocampal Formation , 2020, Current Biology.

[26]  Yong Tao,et al.  Compound facial expressions of emotion , 2014, Proceedings of the National Academy of Sciences.

[27]  Rankin W. McGugin,et al.  The reliability of individual differences in face-selective responses in the fusiform gyrus and their relation to face recognition ability , 2016, Brain Imaging and Behavior.

[28]  H. P. Op de Beeck,et al.  Representations of Facial Identity Information in the Ventral Visual Stream Investigated with Multivoxel Pattern Analyses , 2013, The Journal of Neuroscience.

[29]  Robin A. A. Ince,et al.  Dynamic Construction of Reduced Representations in the Brain for Perceptual Decision Behavior , 2018, Current Biology.

[30]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[31]  C. Jacques,et al.  Early adaptation to repeated unfamiliar faces across viewpoint changes in the right hemisphere: Evidence from the N170 ERP component , 2009, Neuropsychologia.

[32]  Marlene Behrmann,et al.  Feature-based face representations and image reconstruction from behavioral and neural data , 2015, Proceedings of the National Academy of Sciences.

[33]  I. Biederman,et al.  The utility of surface reflectance for the recognition of upright and inverted faces , 2007, Vision Research.

[34]  Brice A. Kuhl,et al.  Reconstructing Perceived and Retrieved Faces from Activity Patterns in Lateral Parietal Cortex , 2016, The Journal of Neuroscience.

[35]  M. Behrmann,et al.  Congenital prosopagnosia without object agnosia? A literature review , 2018, Cognitive neuropsychology.

[36]  Marie L. Smith,et al.  Decoding the dynamic representation of facial expressions of emotion in explicit and incidental tasks , 2019, NeuroImage.

[37]  Kirsten A. Dalrymple,et al.  Spontaneous perceptual facial distortions correlate with ventral occipitotemporal activity , 2014, Neuropsychologia.

[38]  D. Plaut,et al.  Spatiotemporal dynamics of similarity-based neural representations of facial identity , 2016, Proceedings of the National Academy of Sciences.

[39]  J. Gallant,et al.  Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies , 2011, Current Biology.

[40]  Changde Du,et al.  Reconstructing Perceived Images From Human Brain Activities With Bayesian Deep Multiview Learning , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[41]  R. Goebel,et al.  Individual faces elicit distinct response patterns in human anterior temporal cortex , 2007, Proceedings of the National Academy of Sciences.

[42]  Michal Irani,et al.  Convergent evolution of face spaces across human face-selective neuronal groups and deep convolutional networks , 2019, Nature Communications.

[43]  Alice J. O'Toole,et al.  Dissociable Neural Patterns of Facial Identity across Changes in Viewpoint , 2010, Journal of Cognitive Neuroscience.

[44]  Lawrence Sirovich,et al.  Application of the Karhunen-Loeve Procedure for the Characterization of Human Faces , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Sherryse L. Corrow,et al.  Prosopagnosia: current perspectives , 2016, Eye and brain.

[46]  Radoslaw Martin Cichy,et al.  Resolving human object recognition in space and time , 2014, Nature Neuroscience.

[47]  Sander Erik Bosch,et al.  Shared Neural Mechanisms of Visual Perception and Imagery , 2019, Trends in Cognitive Sciences.

[48]  Matthias Niemeier,et al.  The time course of individual face recognition: A pattern analysis of ERP signals , 2016, NeuroImage.

[49]  Antonio Torralba,et al.  Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence , 2016, Scientific Reports.

[50]  Andy C. H. Lee,et al.  Image Reconstruction Reveals the Impact of Aging on Face Perception , 2019, bioRxiv.

[51]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[52]  Shouyu Ling,et al.  How are visual words represented? Insights from EEG‐based visual word decoding, feature derivation and image reconstruction , 2019, Human brain mapping.

[53]  Tomoyasu Horikawa,et al.  Generic decoding of seen and imagined objects using hierarchical visual features , 2015, Nature Communications.

[54]  Markus F. Neumann,et al.  Viewers extract mean and individual identity from sets of famous faces , 2013, Cognition.

[55]  J. D. de Fockert,et al.  Rapid extraction of mean identity from sets of faces. , 2009, Quarterly journal of experimental psychology.

[56]  Nikolaus Kriegeskorte,et al.  Recurrence is required to capture the representational dynamics of the human visual system , 2019, Proceedings of the National Academy of Sciences.

[57]  J. Haxby,et al.  Distributed Neural Systems for Face Perception , 2011 .

[58]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[59]  Nikolaus Kriegeskorte,et al.  Interpreting encoding and decoding models , 2018, Current Opinion in Neurobiology.

[60]  A. Zador,et al.  Neural representation and the cortical code. , 2000, Annual review of neuroscience.

[61]  Swami Sankaranarayanan,et al.  Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms , 2018, Proceedings of the National Academy of Sciences.

[62]  Jesse L. Breedlove,et al.  Generative Feedback Explains Distinct Brain Activity Codes for Seen and Mental Images , 2020, Current Biology.

[63]  John A. Pyles,et al.  Dynamic Encoding of Face Information in the Human Fusiform Gyrus , 2014, Nature Communications.

[64]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[65]  Oliver G. B. Garrod,et al.  Reconstructing dynamic mental models of facial expressions in prosopagnosia reveals distinct representations for identity and expression , 2015, Cortex.

[66]  Jason M Haberman,et al.  Correspondences Rapid extraction of mean emotion and gender from sets of faces , 2007 .

[67]  D. Fiset,et al.  Use of Face Information Varies Systematically From Developmental Prosopagnosics to Super-Recognizers , 2018, Psychological science.

[68]  Bernhard Schölkopf,et al.  Transfer Learning in Brain-Computer Interfaces , 2015, IEEE Computational Intelligence Magazine.

[69]  Matthew F. Peterson,et al.  Individual differences in face-looking behavior generalize from the lab to the world. , 2016, Journal of vision.

[70]  P. Schyns,et al.  A mechanism for impaired fear recognition after amygdala damage , 2005, Nature.

[71]  M. Moscovitch,et al.  The parietal cortex and episodic memory: an attentional account , 2008, Nature Reviews Neuroscience.

[72]  Brice A. Kuhl,et al.  Neural portraits of perception: Reconstructing face images from evoked brain activity , 2014, NeuroImage.

[73]  Andy C. H. Lee,et al.  Going beyond LTM in the MTL: A synthesis of neuropsychological and neuroimaging findings on the role of the medial temporal lobe in memory and perception , 2010, Neuropsychologia.

[74]  Matthias S. Treder,et al.  Evidence that neural information flow is reversed between object perception and object reconstruction from memory , 2018, Nature Communications.

[75]  A. Ruocco,et al.  Facial emotion recognition in borderline personality disorder , 2012, Psychological Medicine.

[76]  R. Goebel,et al.  Neural correlates of shape and surface reflectance information in individual faces , 2009, Neuroscience.

[77]  Jack L. Gallant,et al.  Human Scene-Selective Areas Represent 3D Configurations of Surfaces , 2019, Neuron.

[78]  Doris Y. Tsao,et al.  Comparing face patch systems in macaques and humans , 2008, Proceedings of the National Academy of Sciences.

[79]  M. Tarr,et al.  Internal representations for face detection: An application of noise‐based image classification to BOLD responses , 2013, Human brain mapping.

[80]  W. Freiwald,et al.  Face Processing Systems: From Neurons to Real-World Social Perception. , 2016, Annual review of neuroscience.

[81]  Jörn Diedrichsen,et al.  Peeling the Onion of Brain Representations. , 2019, Annual review of neuroscience.

[82]  Jonathan S. Cant,et al.  A multivariate investigation of visual word, face, and ensemble processing: Perspectives from EEG-based decoding and feature selection. , 2019, Psychophysiology.

[83]  H. Wilson,et al.  fMRI evidence for the neural representation of faces , 2005, Nature Neuroscience.

[84]  Yizhen Zhang,et al.  Neural Encoding and Decoding with Deep Learning for Dynamic Natural Vision , 2016, Cerebral cortex.

[85]  Andy C. H. Lee,et al.  Differentiating the Roles of the Hippocampus and Perirhinal Cortex in Processes beyond Long-Term Declarative Memory: A Double Dissociation in Dementia , 2006, The Journal of Neuroscience.

[86]  M. A. MacIver,et al.  Neuroscience Needs Behavior: Correcting a Reductionist Bias , 2017, Neuron.

[87]  D. Gentner,et al.  Respects for similarity , 1993 .

[88]  Margot J. Taylor,et al.  Inversion and Contrast Polarity Reversal Affect both Encoding and Recognition Processes of Unfamiliar Faces: A Repetition Study Using ERPs , 2002, NeuroImage.

[89]  Doris Y. Tsao,et al.  The Code for Facial Identity in the Primate Brain , 2017, Cell.

[90]  D. Hubel,et al.  Segregation of form, color, movement, and depth: anatomy, physiology, and perception. , 1988, Science.

[91]  H. Wilson,et al.  The neural representation of face space dimensions , 2013, Neuropsychologia.

[92]  K. Grill-Spector,et al.  The human visual cortex. , 2004, Annual review of neuroscience.

[93]  Talia L. Retter,et al.  Face Perception , 2021, Encyclopedia of Evolutionary Psychological Science.

[94]  Oliver G. B. Garrod,et al.  Modelling face memory reveals task-generalizable representations , 2019, Nature Human Behaviour.

[95]  R. Tootell,et al.  An anterior temporal face patch in human cortex, predicted by macaque maps , 2009, Proceedings of the National Academy of Sciences.

[96]  A. Young,et al.  Contributions of feature shapes and surface cues to the recognition and neural representation of facial identity , 2016, Cortex.

[97]  Marcel van Gerven,et al.  Reconstructing perceived faces from brain activations with deep adversarial neural decoding , 2017, NIPS.

[98]  D. Whitney,et al.  Ensemble Perception , 2018, Annual review of psychology.

[99]  B. Rossion,et al.  A Qualitative Impairment in Face Perception in Alzheimer's Disease: Evidence from a Reduced Face Inversion Effect. , 2016, Journal of Alzheimer's disease : JAD.

[100]  T. Palmeri,et al.  Not just the norm: Exemplar-based models also predict face aftereffects , 2014, Psychonomic Bulletin & Review.

[101]  Jean-Baptiste Poline,et al.  Inverse retinotopy: Inferring the visual content of images from brain activation patterns , 2006, NeuroImage.

[102]  Morris Moscovitch,et al.  Face Processing Changes in Normal Aging Revealed by fMRI Adaptation , 2011, Journal of Cognitive Neuroscience.

[103]  Matthias Niemeier,et al.  The Neural Dynamics of Facial Identity Processing: Insights from EEG-Based Pattern Analysis and Image Reconstruction , 2018, eNeuro.

[104]  Vaidehi S. Natu,et al.  The functional neuroanatomy of face perception: from brain measurements to deep neural networks , 2018, Interface Focus.

[105]  Fernando M Ramírez,et al.  Orientation Encoding and Viewpoint Invariance in Face Recognition: Inferring Neural Properties from Large-Scale Signals , 2018, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[106]  Michael B. Lewis,et al.  Face-Space: A Unifying Concept in Face Recognition Research , 2016, Quarterly journal of experimental psychology.

[107]  D. Tsoi,et al.  Is facial emotion recognition impairment in schizophrenia identical for different emotions? A signal detection analysis , 2008, Schizophrenia Research.

[108]  P. Bennett,et al.  Effects of aging on face identification and holistic face processing , 2013, Vision Research.

[109]  N. Kanwisher,et al.  How face perception unfolds over time , 2018, Nature Communications.

[110]  Jonathan S. Cant,et al.  Elucidating the Neural Representation and the Processing Dynamics of Face Ensembles , 2019, The Journal of Neuroscience.

[111]  A. Caramazza,et al.  Decoding representations of face identity that are tolerant to rotation. , 2014, Cerebral cortex.

[112]  Lucy S. Petro,et al.  Dynamics of Visual Information Integration in the Brain for Categorizing Facial Expressions , 2007, Current Biology.

[113]  J. Wilmer Individual Differences in Face Recognition: A Decade of Discovery , 2017 .

[114]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.

[115]  M. Moscovitch,et al.  Broadly tuned face representation in older adults assessed by categorical perception. , 2014, Journal of experimental psychology. Human perception and performance.

[116]  Sarah E. Creighton,et al.  Classification images characterize age-related deficits in face discrimination , 2014, Vision Research.