Combining Sparse and Dense Features to Improve Multi-Modal Registration for Brain DTI Images

A new solution to overcome the constraints of multimodality medical intra-subject image registration is proposed, using the mutual information (MI) of image histogram-oriented gradients as a new matching criterion. We present a rigid, multi-modal image registration algorithm based on linear transformation and oriented gradients for the alignment of T2-weighted (T2w) images (as a fixed reference) and diffusion tensor imaging (DTI) (b-values of 500 and 1250 s/mm2) as floating images of three patients to compensate for the motion during the acquisition process. Diffusion MRI is very sensitive to motion, especially when the intensity and duration of the gradient pulses (characterized by the b-value) increases. The proposed method relies on the whole brain surface and addresses the variability of anatomical features into an image stack. The sparse features refer to corners detected using the Harris corner detector operator, while dense features use all image pixels through the image histogram of oriented gradients (HOG) as a measure of the degree of statistical dependence between a pair of registered images. HOG as a dense feature is focused on the structure and extracts the oriented gradient image in the x and y directions. MI is used as an objective function for the optimization process. The entropy functions and joint entropy function are determined using the HOGs data. To determine the best image transformation, the fiducial registration error (FRE) measure is used. We compare the results against the MI-based intensities results computed using a statistical intensity relationship between corresponding pixels in source and target images. Our approach, which is devoted to the whole brain, shows improved registration accuracy, robustness, and computational cost compared with the registration algorithms, which use anatomical features or regions of interest areas with specific neuroanatomy. Despite the supplementary HOG computation task, the computation time is comparable for MI-based intensities and MI-based HOG methods.

[1]  D. Hill,et al.  Non-rigid image registration: theory and practice. , 2004, The British journal of radiology.

[2]  Maged Goubran,et al.  Multimodal image registration and connectivity analysis for integration of connectomic data from microscopy to MRI , 2019, Nature Communications.

[3]  Zeyun Yu,et al.  Multi-Modal Medical Image Registration with Full or Partial Data: A Manifold Learning Approach , 2018, J. Imaging.

[4]  Olivier D. Faugeras,et al.  Variational Methods for Multimodal Image Matching , 2002, International Journal of Computer Vision.

[5]  Tobias Gass,et al.  Isotropic Total Variation Regularization of Displacements in Parametric Image Registration , 2017, IEEE Transactions on Medical Imaging.

[6]  Bostjan Likar,et al.  Validation for 2D/3D registration. II: The comparison of intensity- and gradient-based merit functions using a new gold standard data set. , 2011, Medical physics.

[7]  Zhe Zhang,et al.  A New Image Registration Algorithm Based on Evidential Reasoning , 2019, Sensors.

[8]  Larry S. Davis,et al.  Vehicle Detection Using Partial Least Squares , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[10]  Xin Li,et al.  Linear Registration of Brain MRI Using Knowledge-Based Multiple Intermediator Libraries , 2019, Front. Neurosci..

[11]  Suprava Patnaik,et al.  Image Registration Using Log Polar Transform and Phase Correlation to Recover Higher Scale , 2012 .

[12]  Emile A. Hendriks,et al.  Confidence Estimation for Medical Image Registration Based On Stereo Confidences , 2016, IEEE Transactions on Medical Imaging.

[13]  Xuming Zhang,et al.  Non-Rigid Registration of Multi-Modality Medical Image Using Combined Gradient Information and Mutual Information , 2018 .

[14]  Shigeki Aoki,et al.  Simultaneous fMRI-EEG-DTI recording of MMN in patients with schizophrenia , 2019, PloS one.

[15]  A Chandler,et al.  Performance Assessment for Brain MR Imaging Registration Methods , 2017, American Journal of Neuroradiology.

[16]  Debiao Li,et al.  Three‐dimensional simultaneous brain T1, T2, and ADC mapping with MR Multitasking , 2019, Magnetic resonance in medicine.

[17]  Agustín Salgado de la Nuez,et al.  An Analysis and Implementation of the Harris Corner Detector , 2018, Image Process. Line.

[18]  J. Michael Fitzpatrick,et al.  General Approach to First-Order Error Prediction in Rigid Point Registration , 2011, IEEE Transactions on Medical Imaging.

[19]  Ahmed Serag,et al.  Histograms of Oriented 3D Gradients for Fully Automated Fetal Brain Localization and Robust Motion Correction in 3 T Magnetic Resonance Images , 2017, BioMed research international.

[20]  Emad Fatemizadeh,et al.  Image Registration Based on Low Rank Matrix: Rank-Regularized SSD , 2018, IEEE Transactions on Medical Imaging.

[21]  R. Lachner,et al.  Enhancing accuracy of magnetic resonance image fusion by defining a volume of interest , 2003, Neuroradiology.

[22]  Wiro J Niessen,et al.  Groupwise image registration based on a total correlation dissimilarity measure for quantitative MRI and dynamic imaging data , 2018, Scientific Reports.

[23]  Leo Joskowicz,et al.  Geometrical analysis of registration errors in point-based rigid-body registration using invariants , 2011, Medical Image Anal..

[24]  Arno Klein,et al.  Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration , 2009, NeuroImage.

[25]  Tiejun Yang,et al.  Nonrigid registration of medical image based on adaptive local structure tensor and normalized mutual information , 2019, Journal of applied clinical medical physics.

[26]  Marc Modat,et al.  Multi-channel registration of fractional anisotropy and T1-weighted images in the presence of atrophy: application to multiple sclerosis. , 2015, Functional neurology.