How can deep learning advance computational modeling of sensory information processing?

Deep learning, computational neuroscience, and cognitive science have overlapping goals related to understanding intelligence such that perception and behaviour can be simulated in computational systems. In neuroimaging, machine learning methods have been used to test computational models of sensory information processing. Recently, these model comparison techniques have been used to evaluate deep neural networks (DNNs) as models of sensory information processing. However, the interpretation of such model evaluations is muddied by imprecise statistical conclusions. Here, we make explicit the types of conclusions that can be drawn from these existing model comparison techniques and how these conclusions change when the model in question is a DNN. We discuss how DNNs are amenable to new model comparison techniques that allow for stronger conclusions to be made about the computational mechanisms underlying sensory information processing.

[1]  Dominic Frigon,et al.  The Mantel test versus Pearson's correlation analysis : Assessment of the differences for biological and environmental studies , 2000 .

[2]  Rainer Goebel,et al.  Editorial: Integrating Computational and Neural Findings in Visual Object Perception , 2016, Front. Comput. Neurosci..

[3]  N. Kriegeskorte,et al.  Author ' s personal copy Representational geometry : integrating cognition , computation , and the brain , 2013 .

[4]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[5]  George Tzanetakis,et al.  Musical genre classification of audio signals , 2002, IEEE Trans. Speech Audio Process..

[6]  D. Poeppel The maps problem and the mapping problem: Two challenges for a cognitive neuroscience of speech and language , 2012, Cognitive neuropsychology.

[7]  Marcel A. J. van Gerven,et al.  A primer on encoding models in sensory neuroscience , 2017 .

[8]  Karl J. Friston,et al.  Comparing the similarity and spatial structure of neural representations: A pattern-component model , 2011, NeuroImage.

[9]  Jörn Diedrichsen,et al.  Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis , 2017, bioRxiv.

[10]  Bertrand Thirion,et al.  How machine learning is shaping cognitive neuroimaging , 2014, GigaScience.

[11]  J. Haynes A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives , 2015, Neuron.

[12]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  G. Buzsáki Rhythms of the brain , 2006 .

[14]  Konrad P. Kording,et al.  Towards an integration of deep learning and neuroscience , 2016, bioRxiv.

[15]  Marcel A. J. van Gerven,et al.  Brains on Beats , 2016, NIPS.

[16]  Rainer Goebel,et al.  Modeling invariant object processing based on tight integration of simulated and empirical data in a Common Brain Space , 2012, Front. Comput. Neurosci..

[17]  Perfecto Herrera-Boyer,et al.  Automatic Classification of Musical Instrument Sounds , 2003 .

[18]  O. Sporns,et al.  Complex brain networks: graph theoretical analysis of structural and functional systems , 2009, Nature Reviews Neuroscience.

[19]  Zachary Chase Lipton The mythos of model interpretability , 2016, ACM Queue.

[20]  J. Gallant,et al.  Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies , 2011, Current Biology.

[21]  Nikolaus Kriegeskorte Deep neural networks: a new framework for modelling biological vision and brain information processing , 2015 .

[22]  Jitendra Malik,et al.  Pixels to Voxels: Modeling Visual Representation in the Human Brain , 2014, ArXiv.

[23]  Jinwen Ma,et al.  MFCC and SVM Based Recognition of Chinese Vowels , 2005, CIS.

[24]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[25]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[26]  Jack L. Gallant,et al.  Encoding and decoding in fMRI , 2011, NeuroImage.

[27]  Nikolaus Kriegeskorte,et al.  Deep neural networks: a new framework for modelling biological vision and brain information processing , 2015, bioRxiv.

[28]  Ha Hong,et al.  Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream , 2013, NIPS.

[29]  Michael S. Pratte,et al.  Integrating Theoretical Models with Functional Neuroimaging. , 2017, Journal of mathematical psychology.

[30]  R. Turner Uses, misuses, new uses and fundamental limitations of magnetic resonance imaging in cognitive science , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[31]  Sean M. Polyn,et al.  Beyond mind-reading: multi-voxel pattern analysis of fMRI data , 2006, Trends in Cognitive Sciences.

[32]  Daniel L. K. Yamins,et al.  Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition , 2014, PLoS Comput. Biol..

[33]  Nikolaus Kriegeskorte,et al.  Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation , 2014, PLoS Comput. Biol..

[34]  J. Gallant,et al.  Complete functional characterization of sensory neurons by system identification. , 2006, Annual review of neuroscience.

[35]  Konrad Paul Kording,et al.  Could a Neuroscientist Understand a Microprocessor? , 2016, bioRxiv.

[36]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[37]  Nikolaus Kriegeskorte,et al.  Relating Population-Code Representations between Man, Monkey, and Computational Models , 2009, Front. Neurosci..

[38]  Konrad P. Körding,et al.  Toward an Integration of Deep Learning and Neuroscience , 2016, bioRxiv.

[39]  Bradley C. Love,et al.  The Algorithmic Level Is the Bridge Between Computation and Brain , 2015, Top. Cogn. Sci..