Big data in nanoscale connectomics, and the greed for training labels

The neurosciences have developed methods that outpace most other biomedical fields in terms of acquired bytes. We review how the information content and analysis challenge of such data indicates that electron microscopy (EM)-based connectomics is an especially hard problem. Here, as in many other current machine learning applications, the need for excessive amounts of labelled data while utilizing only a small fraction of available raw image data for algorithm training illustrates the still fundamental gap between artificial and biological intelligence. Substantial improvements of label and energy efficiency in machine learning may be required to address the formidable challenge of acquiring the nanoscale connectome of a human brain.

[1]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[2]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[3]  M. Konishi,et al.  Axonal delay lines for time measurement in the owl's brainstem. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[4]  M. Schatz,et al.  Big Data: Astronomical or Genomical? , 2015, PLoS biology.

[5]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[6]  Hongkui Zeng,et al.  Neuroinformatics of the Allen Mouse Brain Connectivity Atlas. , 2015, Methods.

[7]  Partha P. Mitra,et al.  The Circuit Architecture of Whole Brains at the Mesoscopic Scale , 2014, Neuron.

[8]  Arthur W. Wetzel,et al.  Network anatomy and in vivo physiology of visual cortical neurons , 2011, Nature.

[9]  Lydia Ng,et al.  The organization of intracortical connections by layer and cell class in the mouse brain , 2018, bioRxiv.

[10]  Sergey L. Gratiy,et al.  Fully integrated silicon probes for high-density recording of neural activity , 2017, Nature.

[11]  S. Mikula Progress Towards Mammalian Whole-Brain Cellular Connectomics , 2016, Front. Neuroanat..

[12]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[13]  Louis K. Scheffer,et al.  A visual motion detection circuit suggested by Drosophila connectomics , 2013, Nature.

[14]  Olaf Sporns,et al.  The Human Connectome: A Structural Description of the Human Brain , 2005, PLoS Comput. Biol..

[15]  Kevin L. Briggman,et al.  Structural and functional diversity of a dense sample of retinal ganglion cells , 2017 .

[16]  Tai Sing Lee,et al.  Large-scale two-photon imaging revealed super-sparse population codes in the V1 superficial layer of awake monkeys , 2018, eLife.

[17]  Philipp J. Keller,et al.  Emerging Imaging and Genomic Tools for Developmental Systems Biology. , 2016, Developmental cell.

[18]  D. Hassabis,et al.  Neuroscience-Inspired Artificial Intelligence , 2017, Neuron.

[19]  Srinivas C. Turaga,et al.  Space-time wiring specificity supports direction selectivity in the retina , 2014, Nature.

[20]  Philipp J. Keller,et al.  Whole-animal functional and developmental imaging with isotropic spatial resolution , 2015, Nature Methods.

[21]  Allan R. Jones,et al.  A mesoscale connectome of the mouse brain , 2014, Nature.

[22]  Patrick van der Smagt,et al.  SynEM, automated synapse detection for connectomics , 2017, eLife.

[23]  Philipp J. Keller,et al.  Real-Time Three-Dimensional Cell Segmentation in Large-Scale Microscopy Data of Developing Embryos. , 2016, Developmental cell.

[24]  William R. Gray Roncal,et al.  Saturated Reconstruction of a Volume of Neocortex , 2015, Cell.

[25]  Daniel R. Berger,et al.  The Fuzzy Logic of Network Connectivity in Mouse Visual Thalamus , 2016, Cell.

[26]  P. Sterling,et al.  How Much the Eye Tells the Brain , 2006, Current Biology.

[27]  G. Urban,et al.  Automated synaptic connectivity inference for volume electron microscopy , 2017, Nature Methods.

[28]  Giulio Tononi,et al.  Ultrastructural evidence for synaptic scaling across the wake/sleep cycle , 2017, Science.

[29]  Kevin L. Briggman,et al.  3D structural imaging of the brain with photons and electrons , 2008, Current Opinion in Neurobiology.

[30]  K. Harris,et al.  Ultrastructural Analysis of Hippocampal Neuropil from the Connectomics Perspective , 2010, Neuron.

[31]  W. Denk,et al.  Serial Block-Face Scanning Electron Microscopy to Reconstruct Three-Dimensional Tissue Nanostructure , 2004, PLoS biology.

[32]  Davi D Bock,et al.  Volume electron microscopy for neuronal circuit reconstruction , 2012, Current Opinion in Neurobiology.

[33]  Nir Shavit,et al.  The big data challenges of connectomics , 2014, Nature Neuroscience.

[34]  Fernando Amat,et al.  Efficient processing and analysis of large-scale light-sheet microscopy data , 2015, Nature Protocols.

[35]  G. Knott,et al.  Ultrastructural analysis of adult mouse neocortex comparing aldehyde perfusion with cryo fixation , 2015, eLife.

[36]  Florian Engert The Big Data Problem: Turning Maps into Knowledge , 2014, Neuron.

[37]  Tapani Raiko,et al.  Semi-supervised Learning with Ladder Networks , 2015, NIPS.

[38]  Moritz Helmstaedter,et al.  The Mutual Inspirations of Machine Learning and Neuroscience , 2015, Neuron.

[39]  T. Sejnowski,et al.  Nanoconnectomic upper bound on the variability of synaptic plasticity , 2015, eLife.

[40]  Pascal Fua,et al.  Learning Context Cues for Synapse Segmentation in EM Volumes , 2012, MICCAI.

[41]  Tom Schaul,et al.  Building Machines that Learn and Think for Themselves: Commentary on Lake et al., Behavioral and Brain Sciences, 2017 , 2017, 1711.08378.

[42]  G. Knott,et al.  Ultrastructurally-smooth thick partitioning and volume stitching for larger-scale connectomics , 2015, Nature Methods.

[43]  J C Fiala,et al.  Reconstruct: a free editor for serial section microscopy , 2005, Journal of microscopy.

[44]  Moritz Helmstaedter,et al.  SegEM: Efficient Image Analysis for High-Resolution Connectomics , 2015, Neuron.

[45]  G. Knott,et al.  Serial Section Scanning Electron Microscopy of Adult Brain Tissue Using Focused Ion Beam Milling , 2008, The Journal of Neuroscience.

[46]  Justin Senseney,et al.  Spatially isotropic four-dimensional imaging with dual-view plane illumination microscopy , 2013, Nature Biotechnology.

[47]  Won-Ki Jeong,et al.  Whole-brain serial-section electron microscopy in larval zebrafish , 2017, Nature.

[48]  Philipp J. Keller,et al.  Fast, high-contrast imaging of animal development with scanned light sheet–based structured-illumination microscopy , 2010, Nature Methods.

[49]  Lorenz Pammer,et al.  Large-scale mapping of cortical synaptic projections with extracellular electrode arrays , 2017, Nature Methods.

[50]  Stephen M. Plaza Focused Proofreading to Reconstruct Neural Connectomes from EM Images at Scale , 2016, LABELS/DLMIA@MICCAI.

[51]  Eric T. Trautman,et al.  A Complete Electron Microscopy Volume of the Brain of Adult Drosophila melanogaster , 2017, Cell.

[52]  Feng Li,et al.  Automated Reconstruction of a Serial-Section EM Drosophila Brain with Flood-Filling Networks and Local Realignment , 2019 .

[53]  Philipp J. Keller,et al.  Whole-brain functional imaging at cellular resolution using light-sheet microscopy , 2013, Nature Methods.

[54]  Jian Sun,et al.  Convolutional neural networks at constrained time cost , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Kevin L. Briggman,et al.  Wiring specificity in the direction-selectivity circuit of the retina , 2011, Nature.

[56]  C. Curcio,et al.  Topography of ganglion cells in human retina , 1990, The Journal of comparative neurology.

[57]  Shin Ishii,et al.  Generative and discriminative model-based approaches to microscopic image restoration and segmentation , 2020, Microscopy.

[58]  Feng Li,et al.  The complete connectome of a learning and memory centre in an insect brain , 2017, Nature.

[59]  Kristin Branson,et al.  A multilevel multimodal circuit enhances action selection in Drosophila , 2015, Nature.

[60]  Jian Sun,et al.  Accelerating Very Deep Convolutional Networks for Classification and Detection , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[62]  Srinivas C. Turaga,et al.  Connectomic reconstruction of the inner plexiform layer in the mouse retina , 2013, Nature.

[63]  Timothy W. Dunn,et al.  Brain-wide mapping of neural activity controlling zebrafish exploratory locomotion , 2016, eLife.

[64]  Amelio Vázquez Reina,et al.  Large-Scale Automatic Reconstruction of Neuronal Processes from Electron Microscopy Images , 2013, Medical Image Anal..

[65]  M. Helmstaedter Cellular-resolution connectomics: challenges of dense neural circuit reconstruction , 2013, Nature Methods.

[66]  M. Konishi,et al.  A circuit for detection of interaural time differences in the brain stem of the barn owl , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[67]  Drew N. Robson,et al.  Brain-wide neuronal dynamics during motor adaptation in zebrafish , 2012, Nature.

[68]  Fred A Hamprecht,et al.  Multicut brings automated neurite segmentation closer to human performance , 2017, Nature Methods.

[69]  Louis K. Scheffer,et al.  A connectome of a learning and memory center in the adult Drosophila brain , 2017, eLife.

[70]  Winfried Denk,et al.  Progress and remaining challenges in high-throughput volume electron microscopy , 2018, Current Opinion in Neurobiology.

[71]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[72]  N. Kasthuri,et al.  Automating the Collection of Ultrathin Serial Sections for Large Volume TEM Reconstructions , 2006, Microscopy and Microanalysis.

[73]  A. Harman,et al.  Neuronal density in the human retinal ganglion cell layer from 16–77 years , 2000, The Anatomical record.

[74]  Ali Farhadi,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016, ECCV.

[75]  M. Helmstaedter,et al.  Dense connectomic reconstruction in layer 4 of the somatosensory cortex , 2018, Science.

[76]  A. L. Eberle,et al.  High-resolution, high-throughput imaging with a multibeam scanning electron microscope , 2015, Journal of microscopy.

[77]  Philipp Otto,et al.  webKnossos: efficient online 3D data annotation for connectomics , 2017, Nature Methods.

[78]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[79]  Larry Lindsey,et al.  High-precision automated reconstruction of neurons with flood-filling networks , 2017, Nature Methods.

[80]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[81]  Winfried Denk,et al.  EM connectomics reveals axonal target variation in a sequence-generating network , 2017, eLife.

[82]  Jeremy D. Schmahmann,et al.  A Proposal for a Coordinated Effort for the Determination of Brainwide Neuroanatomical Connectivity in Model Organisms at a Mesoscopic Scale , 2009, PLoS Comput. Biol..

[83]  W. Denk,et al.  Volume EM Reconstruction of Spinal Cord Reveals Wiring Specificity in Speed-Related Motor Circuits. , 2018, Cell reports.

[84]  A. Wanner,et al.  Dense EM-based reconstruction of the interglomerular projectome in the zebrafish olfactory bulb , 2016, Nature Neuroscience.

[85]  J. Tenenbaum,et al.  Ingredients of intelligence: From classic debates to an engineering roadmap , 2017, Behavioral and Brain Sciences.

[86]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[87]  M. Helmstaedter,et al.  Axonal synapse sorting in medial entorhinal cortex , 2017, Nature.

[88]  Chandan Singh,et al.  Large Scale Image Segmentation with Structured Loss Based Deep Learning for Connectome Reconstruction , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[89]  Brett J. Graham,et al.  Anatomy and function of an excitatory network in the visual cortex , 2016, Nature.

[90]  Moritz Helmstaedter,et al.  High-accuracy neurite reconstruction for high-throughput neuroanatomy , 2011, Nature Neuroscience.