A multimodal computational pipeline for 3D histology of the human brain

Ex vivo imaging enables analysis of the human brain at a level of detail that is not possible in vivo with MRI. In particular, histology can be used to study brain tissue at the microscopic level, using a wide array of different stains that highlight different microanatomical features. Complementing MRI with histology has important applications in ex vivo atlas building and in modeling the link between microstructure and macroscopic MR signal. However, histology requires sectioning tissue, hence distorting its 3D structure, particularly in larger human samples. Here, we present an open-source computational pipeline to produce 3D consistent histology reconstructions of the human brain. The pipeline relies on a volumetric MRI scan that serves as undistorted reference, and on an intermediate imaging modality (blockface photography) that bridges the gap between MRI and histology. We present results on 3D histology reconstruction of a whole human hemisphere.

[1]  Grégoire Malandain,et al.  Fusion of autoradiographs with an MR volume using 2-D and 3-D linear transformations , 2004, NeuroImage.

[2]  Edson Amaro Júnior,et al.  Multimodal Whole Brain Registration: MRI and High Resolution Histology , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[3]  Arthur W. Toga,et al.  Postmortem high-resolution 3-dimensional imaging of the primate brain: Blockface imaging of perfusion stained tissue , 2006, NeuroImage.

[4]  A. Duggento,et al.  MR Imaging-Histology Correlation by Tailored 3D-Printed Slicer in Oncological Assessment , 2019, Contrast media & molecular imaging.

[5]  Thomy Mertzanidou,et al.  VERDICT MRI validation in fresh and fixed prostate specimens using patient‐specific moulds for histological and MR alignment , 2019, NMR in biomedicine.

[6]  Kunie Ando,et al.  3D imaging in the postmortem human brain with CLARITY and CUBIC. , 2018, Handbook of clinical neurology.

[7]  G. Ripandelli,et al.  Optical coherence tomography. , 1998, Seminars in ophthalmology.

[8]  Timo Dickscheid,et al.  High-Resolution Fiber Tract Reconstruction in the Human Brain by Means of Three-Dimensional Polarized Light Imaging , 2011, Front. Neuroinform..

[9]  Marc Modat,et al.  Model-Based Refinement of Nonlinear Registrations in 3D Histology Reconstruction , 2018, MICCAI.

[10]  Eduard Schreibmann,et al.  A Systematic Pipeline for the Objective Comparison of Whole-Brain Spectroscopic MRI with Histology in Biopsy Specimens from Grade 3 Glioma , 2016, Tomography.

[11]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[12]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[13]  Amir Shmuel,et al.  MRI Based Brain-Specific 3D-Printed Model Aligned to Stereotactic Space for Registering Histology to MRI , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[14]  Koenraad Van Leemput,et al.  Fast and sequence-adaptive whole-brain segmentation using parametric Bayesian modeling , 2016, NeuroImage.

[15]  Alan C. Evans,et al.  BigBrain: An Ultrahigh-Resolution 3D Human Brain Model , 2013, Science.

[16]  Hans-Jochen Heinze,et al.  Integration of ultra-high field MRI and histology for connectome based research of brain disorders , 2013, Front. Neuroanat..

[17]  Piotr Klukowski,et al.  A versatile pipeline for the multi-scale digital reconstruction and quantitative analysis of 3 D tissue , 2015 .

[18]  David A. Boas,et al.  Blockface histology with optical coherence tomography: A comparison with Nissl staining , 2014, NeuroImage.

[19]  D. Reich,et al.  Postmortem magnetic resonance imaging to guide the pathologic cut: individualized, 3-dimensionally printed cutting boxes for fixed brains. , 2014, Journal of neuropathology and experimental neurology.

[20]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[21]  Paul A. Viola,et al.  Multi-modal volume registration by maximization of mutual information , 1996, Medical Image Anal..

[22]  Stephen Pickup,et al.  Characterizing the human hippocampus in aging and Alzheimer’s disease using a computational atlas derived from ex vivo MRI and histology , 2018, Proceedings of the National Academy of Sciences.

[23]  Y. Kalaidzidis,et al.  A versatile pipeline for the multi-scale digital reconstruction and quantitative analysis of 3D tissue architecture , 2015, eLife.

[24]  Lewis D. Griffin,et al.  Polarized light imaging of white matter architecture , 2007, Microscopy research and technique.

[25]  Sébastien Ourselin,et al.  Fast free-form deformation using graphics processing units , 2010, Comput. Methods Programs Biomed..

[26]  Nicholas Ayache,et al.  A Log-Euclidean Framework for Statistics on Diffeomorphisms , 2006, MICCAI.

[27]  Vincent Frouin,et al.  Validation of MRI-based 3D digital atlas registration with histological and autoradiographic volumes: An anatomofunctional transgenic mouse brain imaging study , 2010, NeuroImage.

[28]  Nico Karssemeijer,et al.  3D volume reconstruction from serial breast specimen radiographs for mapping between histology and 3D whole specimen imaging , 2017, Medical physics.

[29]  Nikolaus Weiskopf,et al.  Microstructural imaging of human neocortex in vivo , 2018, NeuroImage.

[30]  Tarek Yousry,et al.  Part-to-Whole Registration of Histology and MRI Using Shape Elements , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[31]  M. Helmstaedter,et al.  Dense connectomic reconstruction in layer 4 of the somatosensory cortex , 2018, Science.

[32]  Anne L. Martel,et al.  Reconstruction of 3-dimensional histology volume and its application to study mouse mammary glands. , 2014, Journal of visualized experiments : JoVE.

[33]  Nikos Paragios,et al.  Slice-to-volume medical image registration: a survey , 2017, Medical Image Anal..

[34]  André J. W. van der Kouwe,et al.  A probabilistic atlas of the human thalamic nuclei combining ex vivo MRI and histology , 2018, NeuroImage.

[35]  Eva L. Dyer,et al.  Brain mapping at high resolutions: Challenges and opportunities , 2019 .

[36]  Shunxing Bao,et al.  SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth , 2018, IEEE Transactions on Medical Imaging.

[37]  Warren M. Grill,et al.  Multimodal characterization of the human nucleus accumbens , 2019, NeuroImage.

[38]  Arnauld Sergé,et al.  For3D: Full organ reconstruction in 3D, an automatized tool for deciphering the complexity of lymphoid organs. , 2015, Journal of immunological methods.

[39]  Hei Ming Lai,et al.  Next generation histology methods for three-dimensional imaging of fresh and archival human brain tissues , 2018, Nature Communications.

[40]  Ruopeng Wang,et al.  Polarization sensitive optical coherence microscopy for brain imaging. , 2016, Optics letters.

[41]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[42]  K. Amunts,et al.  Architectonic Mapping of the Human Brain beyond Brodmann , 2015, Neuron.

[43]  Maged Goubran,et al.  Multimodal image registration and connectivity analysis for integration of connectomic data from microscopy to MRI , 2019, Nature Communications.

[44]  Georgios A Keliris,et al.  On the Usage of Brain Atlases in Neuroimaging Research , 2018, Molecular Imaging and Biology.

[45]  Etsuo A. Susaki,et al.  CUBIC pathology: three-dimensional imaging for pathological diagnosis , 2017, Scientific Reports.

[46]  Jeff W Lichtman,et al.  Multicolor multiscale brain imaging with chromatic multiphoton serial microscopy , 2019, Nature Communications.

[47]  Daniel S. Reich,et al.  Custom fit 3D-printed brain holders for comparison of histology with MRI in marmosets , 2016, Journal of Neuroscience Methods.

[48]  C. Hamani,et al.  Magnetic resonance diffusion tensor imaging for the pedunculopontine nucleus: proof of concept and histological correlation , 2017, Brain Structure and Function.

[49]  Max C. Keuken,et al.  Spatial normalization of ultrahigh resolution 7 T magnetic resonance imaging data of the postmortem human subthalamic nucleus: a multistage approach , 2014, Brain Structure and Function.

[50]  Juan Eugenio Iglesias,et al.  Effect of Fluorinert on the Histological Properties of Formalin-Fixed Human Brain Tissue , 2018, Journal of neuropathology and experimental neurology.

[51]  Brian B. Avants,et al.  Histology-derived volumetric annotation of the human hippocampal subfields in postmortem MRI , 2014, NeuroImage.

[52]  G. Ball,et al.  Voxel-wise comparisons of cellular microstructure and diffusion-MRI in mouse hippocampus using 3D Bridging of Optically-clear histology with Neuroimaging Data (3D-BOND) , 2018, Scientific Reports.

[53]  Gamze Altun,et al.  A brief update on physical and optical disector applications and sectioning-staining methods in neuroscience , 2018, Journal of Chemical Neuroanatomy.

[54]  David J. Hawkes,et al.  Microstructural models for diffusion MRI in breast cancer and surrounding stroma: an ex vivo study , 2016, NMR in biomedicine.

[55]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[56]  John P Mugler,et al.  Optimized three‐dimensional fast‐spin‐echo MRI , 2014, Journal of magnetic resonance imaging : JMRI.

[57]  Terry M. Peters,et al.  Image registration of ex-vivo MRI to sparsely sectioned histology of hippocampal and neocortical temporal lobe specimens , 2013, NeuroImage.

[58]  David J. Hawkes,et al.  Apparatus for Histological Validation of In Vivo and Ex Vivo Magnetic Resonance Imaging of the Human Prostate , 2017, Front. Oncol..

[59]  Atsushi Iriki,et al.  A high-throughput neurohistological pipeline for brain-wide mesoscale connectivity mapping of the common marmoset , 2018, bioRxiv.

[60]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[61]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[62]  Wei Min,et al.  Volumetric chemical imaging by clearing-enhanced stimulated Raman scattering microscopy , 2019, Proceedings of the National Academy of Sciences.

[63]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[64]  Shila Ghazanfar,et al.  The human body at cellular resolution: the NIH Human Biomolecular Atlas Program , 2019, Nature.

[65]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[66]  Joe Chalfoun,et al.  MIST: Accurate and Scalable Microscopy Image Stitching Tool with Stage Modeling and Error Minimization , 2017, Scientific Reports.

[67]  Hidekata Hontani,et al.  Robust 3D image reconstruction of pancreatic cancer tumors from histopathological images with different stains and its quantitative performance evaluation , 2019, International Journal of Computer Assisted Radiology and Surgery.

[68]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[69]  Allan R. Jones,et al.  Comprehensive cellular‐resolution atlas of the adult human brain , 2016, The Journal of comparative neurology.

[70]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[71]  Sébastien Ourselin,et al.  A Survey of Methods for 3D Histology Reconstruction , 2018, Medical Image Anal..

[72]  Guy Marchal,et al.  Multi-modality image registration by maximization of mutual information , 1996, Proceedings of the Workshop on Mathematical Methods in Biomedical Image Analysis.

[73]  Maged Goubran,et al.  Magnetic resonance imaging and histology correlation in the neocortex in temporal lobe epilepsy , 2015, Annals of neurology.

[74]  M. Mancini,et al.  Hierarchical Joint Registration of Tissue Blocks With Soft Shape Constraints For Large-Scale Histology of The Human Brain , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[75]  Xia Li,et al.  Accuracy of image registration between MRI and light microscopy in the ex vivo brain. , 2011, Magnetic resonance imaging.

[76]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[77]  Konrad P. Körding,et al.  Quantifying Mesoscale Neuroanatomy Using X-Ray Microtomography , 2016, eNeuro.

[78]  Prasanna Parvathaneni,et al.  Integrating histology and MRI in the first digital brain of common squirrel monkey, Saimiri sciureus , 2015, Medical Imaging.

[79]  Mark Jenkinson,et al.  Dissecting the pathobiology of altered MRI signal in amyotrophic lateral sclerosis: A post mortem whole brain sampling strategy for the integration of ultra-high-field MRI and quantitative neuropathology , 2018, BMC Neuroscience.

[80]  Nicola Palomero-Gallagher,et al.  Cortical layers: Cyto-, myelo-, receptor- and synaptic architecture in human cortical areas , 2017, NeuroImage.

[81]  David T. Jones,et al.  Cascading network failure across the Alzheimer’s disease spectrum , 2015, Brain : a journal of neurology.

[82]  Maik Stille,et al.  3D reconstruction of 2D fluorescence histology images and registration with in vivo MR images: Application in a rodent stroke model , 2013, Journal of Neuroscience Methods.

[83]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[84]  K. Deisseroth,et al.  CLARITY for mapping the nervous system , 2013, Nature Methods.

[85]  Stefan Heldmann,et al.  2D and 3D MALDI-imaging: conceptual strategies for visualization and data mining. , 2014, Biochimica et biophysica acta.

[86]  Marc Dhenain,et al.  High-throughput 3D whole-brain quantitative histopathology in rodents , 2016, Scientific Reports.

[87]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[88]  Anders M. Dale,et al.  Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex , 2001, IEEE Transactions on Medical Imaging.

[89]  Nico Scherf,et al.  Developing 3D microscopy with CLARITY on human brain tissue: Towards a tool for informing and validating MRI-based histology , 2017, NeuroImage.

[90]  Manoel Jacobsen Teixeira,et al.  High thickness histological sections as alternative to study the three-dimensional microscopic human sub-cortical neuroanatomy , 2018, Brain Structure and Function.

[91]  Ben Glocker,et al.  Is Synthesizing MRI Contrast Useful for Inter-modality Analysis? , 2013, MICCAI.