Scalable High-Performance Image Registration Framework by Unsupervised Deep Feature Representations Learning

Feature selection is a critical step in deformable image registration. In particular, selecting the most discriminative features that accurately and concisely describe complex morphological patterns in image patches improves correspondence detection, which in turn improves image registration accuracy. Furthermore, since more and more imaging modalities are being invented to better identify morphological changes in medical imaging data, the development of deformable image registration method that scales well to new image modalities or new image applications with little to no human intervention would have a significant impact on the medical image analysis community. To address these concerns, a learning-based image registration framework is proposed that uses deep learning to discover compact and highly discriminative features upon observed imaging data. Specifically, the proposed feature selection method uses a convolutional stacked autoencoder to identify intrinsic deep feature representations in image patches. Since deep learning is an unsupervised learning method, no ground truth label knowledge is required. This makes the proposed feature selection method more flexible to new imaging modalities since feature representations can be directly learned from the observed imaging data in a very short amount of time. Using the LONI and ADNI imaging datasets, image registration performance was compared to two existing state-of-the-art deformable image registration methods that use handcrafted features. To demonstrate the scalability of the proposed image registration framework, image registration experiments were conducted on 7.0-T brain MR images. In all experiments, the results showed that the new image registration framework consistently demonstrated more accurate registration results when compared to state of the art.

[1]  Daniel Rueckert,et al.  Nonrigid registration using free-form deformations: application to breast MR images , 1999, IEEE Transactions on Medical Imaging.

[2]  Geoffrey E. Hinton A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[3]  Aapo Hyvärinen,et al.  Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces , 2000, Neural Computation.

[4]  Rainer Schrader,et al.  Fast and robust registration of PET and MR images of human brain , 2004, NeuroImage.

[5]  Daniel Rueckert,et al.  LEAP: Learning embeddings for atlas propagation , 2010, NeuroImage.

[6]  Dinggang Shen,et al.  Image registration by local histogram matching , 2007, Pattern Recognit..

[7]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Aapo Hyvärinen,et al.  Independent Component Analysis: Fast ICA by a fixed-point algorithm that maximizes non-Gaussianity , 2001 .

[9]  Paul M. Thompson,et al.  A framework for computational anatomy , 2002 .

[10]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[11]  Paul M. Thompson,et al.  In vivo neuropathology of the hippocampal formation in AD: A radial mapping MR-based study , 2006, NeuroImage.

[12]  A. Toga,et al.  Tracking Alzheimer's Disease , 2007, Annals of the New York Academy of Sciences.

[13]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[14]  Wei Li,et al.  Automatic hippocampus segmentation of 7.0Tesla MR images by combining multiple atlases and auto-context models , 2013, NeuroImage.

[15]  Christos Davatzikos,et al.  Voxel-Based Morphometry Using the RAVENS Maps: Methods and Validation Using Simulated Longitudinal Atrophy , 2001, NeuroImage.

[16]  Dae-Shik Kim,et al.  Quantitative analysis of the hippocampus using images obtained from 7.0 T MRI , 2010, NeuroImage.

[17]  Christos Davatzikos,et al.  GRAM: A framework for geodesic registration on anatomical manifolds , 2010, Medical Image Anal..

[18]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[19]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[20]  C. Jack,et al.  MRI of hippocampal volume loss in early Alzheimer's disease in relation to ApoE genotype and biomarkers , 2008, Brain : a journal of neurology.

[21]  Norbert Schuff,et al.  Alzheimer’s Disease Neuroimaging Initiative: A one-year follow up study using Tensor-Based Morphometry correlating degenerative rates, biomarkers and cognition , 2009 .

[22]  Dinggang Shen,et al.  Attribute Vector Guided Groupwise Registration , 2009, MICCAI.

[23]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[24]  Torsten Rohlfing,et al.  Image Similarity and Tissue Overlaps as Surrogates for Image Registration Accuracy: Widely Used but Unreliable , 2012, IEEE Transactions on Medical Imaging.

[25]  Nicholas Ayache,et al.  A Log-Euclidean Framework for Statistics on Diffeomorphisms , 2006, MICCAI.

[26]  Bernhard Schölkopf,et al.  Learning similarity measure for multi-modal 3D image registration , 2009, CVPR 2009.

[27]  Simon J. Doran,et al.  Stacked Autoencoders for Unsupervised Feature Learning and Multiple Organ Detection in a Pilot Study Using 4D Patient Data , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[29]  A. Toga,et al.  Localizing Age-Related Changes in Brain Structure between Childhood and Adolescence Using Statistical Parametric Mapping , 1999, NeuroImage.

[30]  Honglak Lee,et al.  Sparse deep belief net model for visual area V2 , 2007, NIPS.

[31]  Hervé Delingette,et al.  Registration of 4D Cardiac CT Sequences Under Trajectory Constraints With Multichannel Diffeomorphic Demons , 2010, IEEE Transactions on Medical Imaging.

[32]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[33]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[34]  Dinggang Shen,et al.  HAMMER: hierarchical attribute matching mechanism for elastic registration , 2002, IEEE Transactions on Medical Imaging.

[35]  Alan C. Evans,et al.  Growth patterns in the developing brain detected by using continuum mechanical tensor maps , 2000, Nature.

[36]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Brian B. Avants,et al.  Symmetric Diffeomorphic Image Registration: Evaluating Automated Labeling of Elderly and Neurodegenerative Cortex and Frontal Lobe , 2006, WBIR.

[38]  Arthur W. Toga,et al.  Learning based coarse-to-fine image registration , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Honglak Lee,et al.  Unsupervised learning of hierarchical representations with convolutional deep belief networks , 2011, Commun. ACM.

[40]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[41]  Nikos Paragios,et al.  DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency Weighting , 2009, IPMI.

[42]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[43]  Max A. Viergever,et al.  A survey of medical image registration , 1998, Medical Image Anal..

[44]  Wyeth W. Wasserman,et al.  Deep Feature Selection: Theory and Application to Identify Enhancers and Promoters , 2015, RECOMB.

[45]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[46]  Dinggang Shen,et al.  Learning Best Features and Deformation Statistics for Hierarchical Registration of MR Brain Images , 2007, IPMI.

[47]  Dinggang Shen,et al.  Feature‐based groupwise registration by hierarchical anatomical correspondence detection , 2012, Human brain mapping.

[48]  Kiralee M. Hayashi,et al.  Conversion of mild cognitive impairment to Alzheimer disease predicted by hippocampal atrophy maps. , 2006, Archives of neurology.

[49]  Dinggang Shen,et al.  Deformable segmentation of 3-D ultrasound prostate images using statistical texture matching method , 2006, IEEE Transactions on Medical Imaging.

[50]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[51]  Albert C. S. Chung,et al.  Learning-based non-rigid image registration using prior joint intensity distributions with graph-cuts , 2011, 2011 18th IEEE International Conference on Image Processing.

[52]  Nicholas Ayache,et al.  Understanding the "Demon's Algorithm": 3D Non-rigid Registration by Gradient Descent , 1999, MICCAI.

[53]  Nikolas P. Galatsanos,et al.  A support vector machine approach for detection of microcalcifications , 2002, IEEE Transactions on Medical Imaging.

[54]  Alan C. Evans,et al.  Structural maturation of neural pathways in children and adolescents: in vivo study. , 1999, Science.

[55]  Dinggang Shen,et al.  S‐HAMMER: Hierarchical attribute‐guided, symmetric diffeomorphic registration for MR brain images , 2014, Human brain mapping.

[56]  Shu Liao,et al.  Unsupervised Deep Feature Learning for Deformable Registration of MR Brain Images , 2013, MICCAI.

[57]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[58]  S. Resnick,et al.  Effects of Hormone Replacement Therapy on Cognitive and Brain Aging , 2001, Annals of the New York Academy of Sciences.

[59]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[60]  Karl J. Friston,et al.  Spatial registration and normalization of images , 1995 .

[61]  Arno Klein,et al.  Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration , 2009, NeuroImage.

[62]  Alan C. Evans,et al.  Automated cortical thickness measurements from MRI can accurately separate Alzheimer's patients from normal elderly controls , 2008, Neurobiology of Aging.

[63]  Dinggang Shen,et al.  journal homepage: www.elsevier.com/locate/ynimg , 2022 .

[64]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[65]  Dinggang Shen,et al.  Simulating deformations of MR brain images for validation of atlas-based segmentation and registration algorithms , 2006, NeuroImage.

[66]  Maria Pantoja,et al.  Introduction to Deep Learning , 2018, Deep Learning.

[67]  D. Shen,et al.  Discriminant analysis of longitudinal cortical thickness changes in Alzheimer's disease using dynamic and network features , 2012, Neurobiology of Aging.

[68]  Jae-Yong Han,et al.  New brain atlas—Mapping the human brain in vivo with 7.0 T MRI and comparison with postmortem histology: Will these images change modern medicine? , 2008 .

[69]  Arthur W. Toga,et al.  Construction of a 3D probabilistic atlas of human cortical structures , 2008, NeuroImage.

[70]  Cheng B Saw,et al.  Multimodality image fusion and planning and dose delivery for radiation therapy. , 2008, Medical dosimetry : official journal of the American Association of Medical Dosimetrists.

[71]  Andrew Y. Ng,et al.  Learning Feature Representations with K-Means , 2012, Neural Networks: Tricks of the Trade.

[72]  Quoc V. Le,et al.  On optimization methods for deep learning , 2011, ICML.

[73]  Quoc V. Le,et al.  Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis , 2011, CVPR 2011.

[74]  Tom Vercauteren,et al.  Diffeomorphic demons: Efficient non-parametric image registration , 2009, NeuroImage.

[75]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[76]  Michael Brady,et al.  Saliency, Scale and Image Description , 2001, International Journal of Computer Vision.

[77]  Hervé Delingette,et al.  Registration of 4D Time-Series of Cardiac Images with Multichannel Diffeomorphic Demons , 2008, MICCAI.

[78]  Jayaram K. Udupa,et al.  New methods of MR image intensity standardization via generalized scale , 2005, SPIE Medical Imaging.

[79]  D. Hill,et al.  Non-rigid image registration: theory and practice. , 2004, The British journal of radiology.

[80]  Dinggang Shen,et al.  A General Learning Framework for Non-rigid Image Registration , 2006, MIAR.

[81]  Yoshua Bengio,et al.  An empirical evaluation of deep architectures on problems with many factors of variation , 2007, ICML '07.

[82]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[83]  Rebecca C. Knickmeyer,et al.  A Structural MRI Study of Human Brain Development from Birth to 2 Years , 2008, The Journal of Neuroscience.

[84]  Dinggang Shen,et al.  Learning-based deformable registration of MR brain images , 2006, IEEE Transactions on Medical Imaging.

[85]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[86]  Dinggang Shen,et al.  TPS-HAMMER: Improving HAMMER registration algorithm by soft correspondence matching and thin-plate splines based deformation interpolation , 2010, NeuroImage.

[87]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[88]  Carl-Fredrik Westin,et al.  Improving Registration Using Multi-channel Diffeomorphic Demons Combined with Certainty Maps , 2011, MBIA.

[89]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[90]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[91]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[92]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.