Diffeomorphic registration with intensity transformation and missing data: Application to 3D digital pathology of Alzheimer’s disease

This paper examines the problem of diffeomorphic image mapping in the presence of differing image intensity profiles and missing data. Our motivation comes from the problem of aligning 3D brain MRI with 100 micron isotropic resolution, to histology sections with 1 micron in plane resolution. Multiple stains, as well as damaged, folded, or missing tissue are common in this situation. We overcome these challenges by introducing two new concepts. Cross modality image matching is achieved by jointly estimating polynomial transformations of the atlas intensity, together with pose and deformation parameters. Missing data is accommodated via a multiple atlas selection procedure where several atlases may be of homogeneous intensity and correspond to “background” or “artifact”. The two concepts are combined within an Expectation Maximization algorithm, where atlas selection posteriors and deformation parameters are updated iteratively, and polynomial coefficients are computed in closed form. We show results for 3D reconstruction of digital pathology and MRI in standard atlas coordinates. In conjunction with convolutional neural networks, we quantify the 3D density distribution of tauopathy throughout the medial temporal lobe of an Alzheimer’s disease postmortem specimen. Author summary Our work in Alzheimer’s disease (AD) is attempting to connect histopathology at autopsy and longitudinal clinical magnetic resonance imaging (MRI), combining the strengths of each modality in a common coordinate system. We are bridging this gap by using post mortem high resolution MRI to reconstruct digital pathology in 3D. This image registration problem is challenging because it combines images from different modalities in the presence of missing tissue and artifacts. We overcome this challenge by developing a new registration technique that simultaneously classifies each pixel as “good data” / “missing tissue” / “artifact”, learns a contrast transformation between modalities, and computes deformation parameters. We name this technique “(D)eformable (R)egistration and (I)ntensity (T)ransformation with (M)issing (D)ata”, pronounced as “Dr. It, M.D.”. In conjunction with convolutional neural networks, we use this technique to map the three dimensional distribution of tau tangles in the medial temporal lobe of an AD postmortem specimen.

[1]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[2]  J Talairach,et al.  Application of stereotactic concepts to the surgery of epilepsy. , 1980, Acta neurochirurgica. Supplementum.

[3]  S. M. Sumi,et al.  The Consortium to Establish a Registry for Alzheimer's Disease (CERAD) , 1991, Neurology.

[4]  Terry M. Peters,et al.  3D statistical neuroanatomical models from 305 MRI volumes , 1993, 1993 IEEE Conference Record Nuclear Science Symposium and Medical Imaging Conference.

[5]  M I Miller,et al.  Mathematical textbook of deformable neuroanatomies. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[6]  M. Gearing,et al.  Neuropathologic assessment of Alzheimer's disease , 1997, Neurology.

[7]  R. Saunders,et al.  The entorhinal cortex: an examination of cyto- and myeloarchitectonic organization in humans. , 1997, Cerebral cortex.

[8]  U. Grenander,et al.  Computational anatomy: an emerging discipline , 1998 .

[9]  P. V. van Zijl,et al.  A motion correction scheme by twin‐echo navigation for diffusion‐weighted magnetic resonance imaging with multiple RF echo acquisition , 1998, Magnetic resonance in medicine.

[10]  Gary E. Christensen,et al.  Consistent image registration , 2001, IEEE Transactions on Medical Imaging.

[11]  Chris Rorden,et al.  Spatial Normalization of Brain Images with Focal Lesions Using Cost Function Masking , 2001, NeuroImage.

[12]  A W Toga,et al.  Maps of the Brain , 2001, The Anatomical record.

[13]  P. V. van Zijl,et al.  Rapid three‐dimensional diffusion MRI facilitates the study of acute stroke in mice , 2001, Magnetic resonance in medicine.

[14]  Michael J Ackerman,et al.  Engineering and algorithm design for an image processing Api: a technical report on ITK--the Insight Toolkit. , 2002, Studies in health technology and informatics.

[15]  L. Younes,et al.  On the metrics and euler-lagrange equations of computational anatomy. , 2002, Annual review of biomedical engineering.

[16]  Paul M. Thompson,et al.  A framework for computational anatomy , 2002 .

[17]  David R. Haynor,et al.  PET-CT image registration in the chest using free-form deformations , 2003, IEEE Transactions on Medical Imaging.

[18]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[19]  Takashi Asada,et al.  [Diagnosis of mild cognitive impairment]. , 2004, Seishin shinkeigaku zasshi = Psychiatria et neurologia Japonica.

[20]  Nicholas Ayache,et al.  Non-rigid Atlas to Subject Registration with Pathologies for Conformal Brain Radiotherapy , 2004, MICCAI.

[21]  Alain Trouvé,et al.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms , 2005, International Journal of Computer Vision.

[22]  H. Braak,et al.  Neuropathological stageing of Alzheimer-related changes , 2004, Acta Neuropathologica.

[23]  S. Wakana,et al.  MRI Atlas of Human White Matter , 2005 .

[24]  P. Szeszko,et al.  MRI atlas of human white matter , 2006 .

[25]  Hany Farid,et al.  Medical image registration with partial data , 2006, Medical Image Anal..

[26]  Brian B. Avants,et al.  Symmetric Diffeomorphic Image Registration: Evaluating Automated Labeling of Elderly and Neurodegenerative Cortex and Frontal Lobe , 2006, WBIR.

[27]  H. Braak,et al.  Staging of Alzheimer disease-associated neurofibrillary pathology using paraffin sections and immunocytochemistry , 2006, Acta Neuropathologica.

[28]  Karl J. Friston,et al.  Statistical parametric mapping , 2013 .

[29]  Michael I. Miller,et al.  Pattern Theory: From Representation to Inference , 2007 .

[30]  Alain Trouvé,et al.  Bayesian template estimation in computational anatomy , 2008, NeuroImage.

[31]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[32]  Dinggang Shen,et al.  ORBIT: A Multiresolution Framework for Deformable Registration of Brain Tumor Images , 2008, IEEE Transactions on Medical Imaging.

[33]  Alain Trouvé,et al.  A Forward Model to Build Unbiased Atlases from Curves and Surfaces , 2008 .

[34]  M. Sdika,et al.  Nonrigid registration of multiple sclerosis brain images using lesion inpainting for morphometry or lesion mapping , 2009, Human brain mapping.

[35]  Eigil Samset,et al.  A Non-rigid Registration Framework That Accommodates Resection and Retraction , 2009, IPMI.

[36]  John Ashburner,et al.  Computational anatomy with the SPM software. , 2009, Magnetic resonance imaging.

[37]  Laurent Younes,et al.  Template registration with missing parts: Application to the segmentation of M. tuberculosis infected lungs , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[38]  Arthur W. Toga,et al.  Atlas-guided tract reconstruction for automated and comprehensive examination of the white matter anatomy , 2010, NeuroImage.

[39]  John S. Duncan,et al.  Non-rigid Registration with Missing Correspondences in Preoperative and Postresection Brain Images , 2010, MICCAI.

[40]  Eigil Samset,et al.  Validation of a nonrigid registration framework that accommodates tissue resection , 2010, Medical Imaging.

[41]  James S. Duncan,et al.  Pairwise registration of images with missing correspondences due to resection , 2010, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[42]  Daniel Rueckert,et al.  Coronary Motion Estimation from CTA Using Probability Atlas and Diffeomorphic Registration , 2010, MIAR.

[43]  Daniel Rueckert,et al.  Diffeomorphic Atlas Estimation using Karcher Mean and Geodesic Shooting on Volumetric Images , 2011, MIUA.

[44]  Xin Xu,et al.  An MRI-based atlas and database of the developing mouse brain , 2011, NeuroImage.

[45]  Nick C Fox,et al.  The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer's disease , 2011, Alzheimer's & Dementia.

[46]  Michael I. Miller,et al.  Matching Sparse Sets of Cardiac Image Cross-Sections Using Large Deformation Diffeomorphic Metric Mapping Algorithm , 2011, STACOM.

[47]  Nick C Fox,et al.  The Diagnosis of Mild Cognitive Impairment due to Alzheimer’s Disease: Recommendations from the National Institute on Aging-Alzheimer’s Association Workgroups on Diagnostic Guidelines for Alzheimer’s Disease , 2011 .

[48]  J. Schneider,et al.  National Institute on Aging–Alzheimer's Association guidelines for the neuropathologic assessment of Alzheimer's disease , 2012, Alzheimer's & Dementia.

[49]  Nassir Navab,et al.  Entropy and Laplacian images: Structural representations for multi-modal registration , 2012, Medical Image Anal..

[50]  Xiaoxing Li,et al.  Registration of Images With Varying Topology Using Embedded Maps , 2012, IEEE Transactions on Medical Imaging.

[51]  Sebastian Schafer,et al.  Extra-dimensional Demons: a method for incorporating missing tissue in deformable image registration. , 2012, Medical physics.

[52]  Michael Brady,et al.  MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration , 2012, Medical Image Anal..

[53]  Paul A. Yushkevich,et al.  Multi-Atlas Segmentation with Joint Label Fusion , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Susumu Mori,et al.  Atlas-based neuroinformatics via MRI: harnessing information from past clinical cases and quantitative image analysis for patient care. , 2013, Annual review of biomedical engineering.

[55]  David S. Lee,et al.  The diffeomorphometry of temporal lobe structures in preclinical Alzheimer's disease☆ , 2013, NeuroImage: Clinical.

[56]  Aaron S. Andalman,et al.  Structural and molecular interrogation of intact biological systems , 2013, Nature.

[57]  Lydia Ng,et al.  Allen Brain Atlas: an integrated spatio-temporal portal for exploring the central nervous system , 2012, Nucleic Acids Res..

[58]  Michael I. Miller,et al.  High-throughput neuro-imaging informatics , 2013, Front. Neuroinform..

[59]  Xiaoying Tang,et al.  Bayesian Parameter Estimation and Segmentation in the Multi-Atlas Random Orbit Model , 2013, PloS one.

[60]  Nancy Wilkins-Diehr,et al.  XSEDE: Accelerating Scientific Discovery , 2014, Computing in Science & Engineering.

[61]  Brian B. Avants,et al.  Histology-derived volumetric annotation of the human hippocampal subfields in postmortem MRI , 2014, NeuroImage.

[62]  L. Younes,et al.  Inferring changepoint times of medial temporal lobe morphometric change in preclinical Alzheimer's disease , 2014, NeuroImage: Clinical.

[63]  L. Younes,et al.  Diffeomorphometry and geodesic positioning systems for human anatomy. , 2014, Technology.

[64]  N. Renier,et al.  iDISCO: A Simple, Rapid Method to Immunolabel Large Tissue Samples for Volume Imaging , 2014, Cell.

[65]  J. Gee,et al.  The Insight ToolKit image registration framework , 2014, Front. Neuroinform..

[66]  Michael I. Miller,et al.  Network Neurodegeneration in Alzheimer’s Disease via MRI Based Shape Diffeomorphometry and High-Field Atlasing , 2015, Front. Bioeng. Biotechnol..

[67]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[68]  Paul W. Frankland,et al.  Optimization of CLARITY for Clearing Whole-Brain and Other Intact Organs1,2,3 , 2015, eNeuro.

[69]  Thorsten Gerber,et al.  Human Nervous System , 2016 .

[70]  Andrea Bergmann,et al.  Statistical Parametric Mapping The Analysis Of Functional Brain Images , 2016 .

[71]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[72]  M. Albert,et al.  Entorhinal and transentorhinal atrophy in mild cognitive impairment using longitudinal diffeomorphometry , 2017, Alzheimer's & dementia.

[73]  Jordan Matelsky,et al.  A Large Deformation Diffeomorphic Approach to Registration of CLARITY Images via Mutual Information , 2017, MICCAI.

[74]  Xiaoying Tang,et al.  Fast diffeomorphic image registration via GPU-based parallel computing: an investigation of the matching cost function , 2018, Medical Imaging.

[75]  Jan Kybic,et al.  Benchmarking of Image Registration Methods for Differently Stained Histological Slides , 2018, 2018 25th IEEE International Conference on Image Processing (ICIP).

[76]  L. Younes,et al.  Identifying Changepoints in Biomarkers During the Preclinical Phase of AD , 2018 .

[77]  Sébastien Ourselin,et al.  A Survey of Methods for 3D Histology Reconstruction , 2018, Medical Image Anal..

[78]  Alzheimer's Disease Neuroimaging Initiative,et al.  Cortical thickness atrophy in the transentorhinal cortex in mild cognitive impairment , 2018, NeuroImage: Clinical.

[79]  Partha P. Mitra,et al.  On variational solutions for whole brain serial-section histology using the computational anatomy random orbit model , 2018, bioRxiv.

[80]  Daniel J. Tward,et al.  QUANTIFICATION OF 3D TANGLE DISTRIBUTION IN MEDIAL TEMPORAL LOBE USING MULTIMODAL IMAGE REGISTRATION AND CONVOLUTIONAL NEURAL NETWORKS , 2018, Alzheimer's & Dementia.

[81]  Michael I Miller,et al.  Computational anatomy and diffeomorphometry: A dynamical systems model of neuroanatomy in the soft condensed matter continuum , 2018, Wiley interdisciplinary reviews. Systems biology and medicine.

[82]  J. Troncoso,et al.  QUANTIFICATION OF 3D TANGLE DISTRIBUTION IN MEDIAL TEMPORAL LOBE USING MULTIMODAL IMAGE REGISTRATION AND CONVOLUTIONAL NEURAL NETWORKS , 2018, Alzheimer's & Dementia.

[83]  Fereshteh S. Bashiri,et al.  Multi-Modal Medical Image Registration with Full or Partial Data: A Manifold Learning Approach , 2018, J. Imaging.

[84]  Michael I. Miller,et al.  Diffeomorphic Upsampling of Serially Acquired Sparse 2D Cross-Sections in Cardiac MRI , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[85]  Sune Darkner,et al.  TopAwaRe: Topology-Aware Registration , 2019, MICCAI.