On simplicity and complexity in the brave new world of large-scale neuroscience

Technological advances have dramatically expanded our ability to probe multi-neuronal dynamics and connectivity in the brain. However, our ability to extract a simple conceptual understanding from complex data is increasingly hampered by the lack of theoretically principled data analytic procedures, as well as theoretical frameworks for how circuit connectivity and dynamics can conspire to generate emergent behavioral and cognitive functions. We review and outline potential avenues for progress, including new theories of high dimensional data analysis, the need to analyze complex artificial networks, and methods for analyzing entire spaces of circuit models, rather than one model at a time. Such interplay between experiments, data analysis and theory will be indispensable in catalyzing conceptual advances in the age of large-scale neuroscience.

[1]  Stephen J. Smith,et al.  Array Tomography: A New Tool for Imaging the Molecular Architecture and Ultrastructure of Neural Circuits , 2007, Neuron.

[2]  Razvan Pascanu,et al.  On the number of inference regions of deep feed forward networks with piece-wise linear activations , 2013, ICLR.

[3]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[4]  N. Wingreen,et al.  Emergence of Preferred Structures in a Simple Model of Protein Folding , 1996, Science.

[5]  Jianfeng Yao,et al.  A note on a Marčenko–Pastur type theorem for time series , 2011, 1109.1612.

[6]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[7]  Kae Nakamura,et al.  Coding of Task Reward Value in the Dorsal Raphe Nucleus , 2010, The Journal of Neuroscience.

[8]  H. Seung,et al.  Serial two-photon tomography: an automated method for ex-vivo mouse brain imaging , 2011, Nature Methods.

[9]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[10]  L. Abbott,et al.  Cascade Models of Synaptically Stored Memories , 2005, Neuron.

[11]  Alan Carleton,et al.  Dynamic Ensemble Odor Coding in the Mammalian Olfactory Bulb: Sensory Information at Different Timescales , 2008, Neuron.

[12]  E. Miller,et al.  Task-Dependent Changes in Short-Term Memory in the Prefrontal Cortex , 2010, The Journal of Neuroscience.

[13]  Jitendra Malik,et al.  Pixels to Voxels: Modeling Visual Representation in the Human Brain , 2014, ArXiv.

[14]  Raj Rao Nadakuditi,et al.  The singular values and vectors of low rank perturbations of large rectangular random matrices , 2011, J. Multivar. Anal..

[15]  Luca Maria Gambardella,et al.  Deep Neural Networks Segment Neuronal Membranes in Electron Microscopy Images , 2012, NIPS.

[16]  Andreas T Schaefer,et al.  Transfection via whole-cell recording in vivo: bridging single-cell physiology, genetics and connectomics , 2011, Nature Neuroscience.

[17]  Ha Hong,et al.  Performance-optimized hierarchical models predict neural responses in higher visual cortex , 2014, Proceedings of the National Academy of Sciences.

[18]  Surya Ganguli,et al.  Memory traces in dynamical systems , 2008, Proceedings of the National Academy of Sciences.

[19]  M. Laubach,et al.  Delay activity in rodent frontal cortex during a simple reaction time task. , 2009, Journal of neurophysiology.

[20]  J. Hegdé,et al.  Temporal dynamics of shape analysis in macaque visual area V2. , 2004, Journal of neurophysiology.

[21]  Baranidharan Raman,et al.  Temporally Diverse Firing Patterns in Olfactory Receptor Neurons Underlie Spatiotemporal Neural Codes for Odors , 2010, The Journal of Neuroscience.

[22]  G. Shepherd.  The Synaptic Organization of the Brain , 1979 .

[23]  N. Matsuki,et al.  Metastability of Active CA3 Networks , 2007, The Journal of Neuroscience.

[24]  David L. Donoho,et al.  Optimal Shrinkage of Singular Values , 2014, IEEE Transactions on Information Theory.

[25]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[26]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[27]  Joel A. Tropp,et al.  Living on the edge: phase transitions in convex programs with random data , 2013, 1303.6672.

[28]  M. Khamassi,et al.  Replay of rule-learning related neural patterns in the prefrontal cortex during sleep , 2009, Nature Neuroscience.

[29]  Jacob T. Robinson,et al.  Vertical nanowire electrode arrays as a scalable platform for intracellular interfacing to neuronal circuits. , 2012, Nature nanotechnology.

[30]  James L. McClelland,et al.  Learning hierarchical category structure in deep neural networks , 2013 .

[31]  Sanjoy Dasgupta,et al.  An elementary proof of a theorem of Johnson and Lindenstrauss , 2003, Random Struct. Algorithms.

[32]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[33]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[35]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[37]  Surya Ganguli,et al.  Identifying and attacking the saddle point problem in high-dimensional non-convex optimization , 2014, NIPS.

[38]  E. Boyden,et al.  Simultaneous whole-animal 3D-imaging of neuronal activity using light-field microscopy , 2014, Nature Methods.

[39]  Lacey J. Kitch,et al.  Long-term dynamics of CA1 hippocampal place codes , 2013, Nature Neuroscience.

[40]  E. Marder,et al.  Similar network activity from disparate circuit parameters , 2004, Nature Neuroscience.

[41]  S. Yamane,et al.  Population dynamics of face-responsive neurons in the inferior temporal cortex. , 2005, Cerebral cortex.

[42]  Daniel L. K. Yamins,et al.  Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition , 2014, PLoS Comput. Biol..

[43]  R. Prevedel,et al.  Brain-wide 3D imaging of neuronal activity in Caenorhabditis elegans with sculpted light , 2013, Nature Methods.

[44]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[45]  Andrea Montanari,et al.  The phase transition of matrix recovery from Gaussian measurements matches the minimax MSE of matrix denoising , 2013, Proceedings of the National Academy of Sciences.

[46]  Eve Marder,et al.  Correlations in ion channel expression emerge from homeostatic tuning rules , 2013, Proceedings of the National Academy of Sciences.

[47]  Arthur W. Wetzel,et al.  Network anatomy and in vivo physiology of visual cortical neurons , 2011, Nature.

[48]  Jean-Marc Goaillard,et al.  Quantitative expression profiling of identified neurons reveals cell-specific constraints on highly variable levels of gene expression , 2007, Proceedings of the National Academy of Sciences.

[49]  V. Jayaraman,et al.  Intensity versus Identity Coding in an Olfactory System , 2003, Neuron.

[50]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[51]  Louis K. Scheffer,et al.  A visual motion detection circuit suggested by Drosophila connectomics , 2013, Nature.

[52]  Drew N. Robson,et al.  Brain-wide neuronal dynamics during motor adaptation in zebrafish , 2012, Nature.

[53]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[54]  Christian K. Machens,et al.  Behavioral / Systems / Cognitive Functional , But Not Anatomical , Separation of “ What ” and “ When ” in Prefrontal Cortex , 2009 .

[55]  H. Sompolinsky,et al.  Compressed sensing, sparsity, and dimensionality in neuronal information processing and data analysis. , 2012, Annual review of neuroscience.

[56]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[57]  Franco Scarselli,et al.  On the Complexity of Neural Network Classifiers: A Comparison Between Shallow and Deep Architectures , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[58]  Byron M. Yu,et al.  Mixture of Trajectory Models for Neural Decoding of Goal-directed Movements a Computational Model of Craving and Obsession Decoding Visual Inputs from Multiple Neurons in the Human Temporal Lobe Encoding Contribution of Individual Retinal Ganglion Cell Responses to Velocity and Acceleration , 2008 .

[59]  Surya Ganguli,et al.  Statistical mechanics of compressed sensing. , 2010, Physical review letters.

[60]  Allan R. Jones,et al.  A mesoscale connectome of the mouse brain , 2014, Nature.

[61]  Q. Ouyang,et al.  The yeast cell-cycle network is robustly designed. , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[62]  N S Wingreen,et al.  Are protein folds atypical? , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[63]  Ian R. Wickersham,et al.  Monosynaptic Restriction of Transsynaptic Tracing from Single, Genetically Targeted Neurons , 2007, Neuron.

[64]  B. Frey,et al.  The human splicing code reveals new insights into the genetic determinants of disease , 2015, Science.

[65]  Surya Ganguli,et al.  Short-term memory in neuronal networks through dynamical compressed sensing , 2010, NIPS.

[66]  B. Nadler,et al.  Global Features of Neural Activity in the Olfactory System Form a Parallel Code That Predicts Olfactory Behavior and Perception , 2010, The Journal of Neuroscience.

[67]  James C Harris,et al.  The Tempest. , 2007, Archives of general psychiatry.

[68]  G. Laurent,et al.  Adaptive regulation of sparseness by feedforward inhibition , 2007, Nature Neuroscience.

[69]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[70]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[71]  Konrad P Kording,et al.  How advances in neural recording affect data analysis , 2011, Nature Neuroscience.

[72]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[73]  M. Nicolelis,et al.  Principal component analysis of neuronal ensemble activity reveals multidimensional somatosensory representations , 1999, Journal of Neuroscience Methods.

[74]  Surya Ganguli,et al.  A memory frontier for complex synapses , 2013, NIPS.

[75]  Matthew T. Kaufman,et al.  Neural population dynamics during reaching , 2012, Nature.

[76]  Hao Li,et al.  Designability and thermal stability of protein structures , 2003, cond-mat/0303600.

[77]  K. Deisseroth,et al.  CLARITY for mapping the nervous system , 2013, Nature Methods.

[78]  Richard G. Baraniuk,et al.  Random Projections of Smooth Manifolds , 2009, Found. Comput. Math..

[79]  Norio Matsuki,et al.  Metastability of Active CA 3 Networks , 2007 .

[80]  Haim Sompolinsky,et al.  Short-term memory in orthogonal neural networks. , 2004, Physical review letters.

[81]  G. Laurent,et al.  Transient Dynamics versus Fixed Points in Odor Representations by Locust Antennal Lobe Projection Neurons , 2005, Neuron.

[82]  Surya Ganguli,et al.  Function constrains network architecture and dynamics: a case study on the yeast cell cycle Boolean network. , 2006, Physical review. E, Statistical, nonlinear, and soft matter physics.

[83]  Hagai Bergman,et al.  Emerging Patterns of Neuronal Responses in Supplementary and Primary Motor Areas during Sensorimotor Adaptation , 2005, The Journal of Neuroscience.

[84]  A. Nica,et al.  On the multiplication of free N-tuples of noncommutative random variables , 1996, funct-an/9604011.

[85]  S. Ganguli,et al.  Statistical mechanics of complex neural systems and high dimensional data , 2013, 1301.7115.