Generating Anthropomorphic Phantoms Using Fully Unsupervised Deformable Image Registration with Convolutional Neural Networks.

PURPOSE Computerized phantoms have been widely used in nuclear medicine imaging for imaging system optimization and validation. Although the existing computerized phantoms can model anatomical variations through organ and phantom scaling, they do not provide a way to fully reproduce the anatomical variations and details seen in humans. In this work, we present a novel registration-based method for creating highly anatomically detailed computerized phantoms. We experimentally show substantially improved image similarity of the generated phantom to a patient image. METHODS We propose a deep-learning-based unsupervised registration method to generate a highly anatomically detailed computerized phantom by warping an XCAT phantom to a patient CT scan. We implemented and evaluated the proposed method using the NURBS-based XCAT phantom and a publicly available low-dose CT dataset from TCIA. A rigorous tradeoff analysis between image similarity and deformation regularization was conducted to select the loss function and regularization term for the proposed method. A novel SSIM-based unsupervised objective function was proposed. Finally, ablation studies were conducted to evaluate the performance of the proposed method (using the optimal regularization and loss function) and the current state-of-the-art unsupervised registration methods. RESULTS The proposed method outperformed the state-of-the-art registration methods, such as SyN and VoxelMorph, by more than 8%, measured by the SSIM and less than 30%, by the MSE. The phantom generated by the proposed method was highly detailed and was almost identical in appearance to a patient image. CONCLUSIONS A deep-learning-based unsupervised registration method was developed to create anthropomorphic phantoms with anatomies labels that can be used as the basis for modeling organ properties. Experimental results demonstrate the effectiveness of the proposed method. The resulting anthropomorphic phantom is highly realistic. Combined with realistic simulations of the image formation process, the generated phantoms could serve in many applications of medical imaging research.

[1]  Hervé Delingette,et al.  Robust Non-rigid Registration Through Agent-Based Action Learning , 2017, MICCAI.

[2]  Katsuyuki Taguchi,et al.  Estimation of Basis Line-Integrals in a Spectral Distortion-Modeled Photon Counting Detector Using Low-Rank Approximation-Based X-Ray Transmittance Modeling: K-Edge Imaging Application , 2017, IEEE Transactions on Medical Imaging.

[3]  Katsuyuki Taguchi,et al.  Joint estimation of tissue types and linear attenuation coefficients for photon counting CT. , 2015, Medical physics.

[4]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[5]  Alain Trouvé,et al.  Computing Large Deformation Metric Mappings via Geodesic Flows of Diffeomorphisms , 2005, International Journal of Computer Vision.

[6]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.

[7]  Eric C Frey,et al.  A Monte Carlo and physical phantom evaluation of quantitative In-111 SPECT , 2005, Physics in medicine and biology.

[8]  Max A. Viergever,et al.  A deep learning framework for unsupervised affine and deformable image registration , 2018, Medical Image Anal..

[9]  Daguang Xu,et al.  NeurReg: Neural Registration and Its Application to Image Segmentation , 2019, 2020 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  David R. Haynor,et al.  PET-CT image registration in the chest using free-form deformations , 2003, IEEE Transactions on Medical Imaging.

[11]  Michael Unser,et al.  On the asymptotic convergence of B-spline wavelets to Gabor functions , 1992, IEEE Trans. Inf. Theory.

[12]  Boudewijn P. F. Lelieveldt,et al.  Nonrigid Image Registration Using Multi-scale 3D Convolutional Neural Networks , 2017, MICCAI.

[13]  Seyed-Mohsen Moosavi-Dezfooli,et al.  DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Mert R. Sabuncu,et al.  VoxelMorph: A Learning Framework for Deformable Medical Image Registration , 2018, IEEE Transactions on Medical Imaging.

[15]  H. Malcolm Hudson,et al.  Accelerated image reconstruction using ordered subsets of projection data , 1994, IEEE Trans. Medical Imaging.

[16]  Thomas Sangild Sørensen,et al.  Registration-Based Reconstruction of Four-Dimensional Cone Beam Computed Tomography , 2013, IEEE Transactions on Medical Imaging.

[17]  Laurent Risser,et al.  An implicit sliding-motion preserving regularisation via bilateral filtering for deformable image registration , 2014, Medical Image Anal..

[18]  Jongha Lee,et al.  View-interpolation of sparsely sampled sinogram using convolutional neural network , 2017, Medical Imaging.

[19]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[20]  Michael Ljungberg,et al.  Monte Carlo Calculation in Nuclear Medicine: Applications in Diagnostic Imaging , 2012 .

[21]  Alan C. Evans,et al.  PET-SORTEO: a Monte Carlo-based Simulator with high count rate capabilities , 2004, IEEE Transactions on Nuclear Science.

[22]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[23]  Baris Turkbey,et al.  The Kinetics and Reproducibility of 18F-Sodium Fluoride for Oncology Using Current PET Camera Technology , 2012, The Journal of Nuclear Medicine.

[24]  W P Segars,et al.  Realistic CT simulation using the 4D XCAT phantom. , 2008, Medical physics.

[25]  Eric C Frey,et al.  An SVD Investigation of Modeling Scatter in Multiple Energy Windows for Improved SPECT Images. , 1996, IEEE transactions on nuclear science.

[26]  Eric C Frey,et al.  Collimator optimization in myocardial perfusion SPECT using the ideal observer and realistic background variability for lesion detection and joint detection and localization tasks , 2016, Physics in medicine and biology.

[27]  Tanya Schmah,et al.  FAIM - A ConvNet Method for Unsupervised 3D Medical Image Registration , 2018, MLMI@MICCAI.

[28]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Daniel Rueckert,et al.  Diffeomorphic Registration Using B-Splines , 2006, MICCAI.

[30]  H Zaidi,et al.  Contourlet-based active contour model for PET image segmentation. , 2013, Medical physics.

[31]  Danielle F. Pace,et al.  A Locally Adaptive Regularization Based on Anisotropic Diffusion for Deformable Image Registration of Sliding Organs , 2013, IEEE Transactions on Medical Imaging.

[32]  Jan Schuemann.  Monte Carlo Calculations in Nuclear Medicine, Second Edition: Applications in Diagnostic Imaging. , 2014 .

[33]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[34]  E. Frey,et al.  A practical method for incorporating scatter in a projector-backprojector for accurate scatter compensation in SPECT , 1993 .

[35]  Jianhua Ma,et al.  A new CT reconstruction technique using adaptive deformation recovery and intensity correction (ADRIC) , 2017, Medical physics.

[36]  Katsuyuki Taguchi,et al.  Estimation of Basis Line-Integrals in a Spectral Distortion-Modeled Photon Counting Detector Using Low-Order Polynomial Approximation of X-ray Transmittance , 2017, IEEE Trans. Medical Imaging.

[37]  S Stute,et al.  GATE V6: a major enhancement of the GATE simulation platform enabling modelling of CT and radiotherapy , 2011, Physics in medicine and biology.

[38]  Max A. Viergever,et al.  End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network , 2017, DLMIA/ML-CDS@MICCAI.

[39]  Eric C Frey,et al.  Current Pediatric Administered Activity Guidelines for 99m Tc-DMSA SPECT Based on Patient Weight Do Not Provide the Same Task-based Image Quality. , 2019, Medical physics.

[40]  Eric C. Frey,et al.  Combination of MCNP and SimSET for Monte Carlo simulation of SPECT with medium and high energy photons , 2001, 2001 IEEE Nuclear Science Symposium Conference Record (Cat. No.01CH37310).

[41]  Yong Fan,et al.  Non-rigid image registration using self-supervised fully convolutional networks without training data , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[42]  Eric C. Frey,et al.  Incorporating CT prior information in the robust fuzzy C-means algorithm for QSPECT image segmentation , 2019, Medical Imaging: Image Processing.

[43]  Tobias Gass,et al.  Isotropic Total Variation Regularization of Displacements in Parametric Image Registration , 2017, IEEE Transactions on Medical Imaging.

[44]  Ananthram Swami,et al.  The Limitations of Deep Learning in Adversarial Settings , 2015, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[45]  Yong Du,et al.  Comparison of Residence Time Estimation Methods for Radioimmunotherapy Dosimetry and Treatment Planning—Monte Carlo Simulation Studies , 2008, IEEE Transactions on Medical Imaging.

[46]  George Wolberg,et al.  Robust image registration using log-polar transform , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[47]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[48]  B. F. Logan,et al.  The Fourier reconstruction of a head section , 1974 .

[49]  Hervé Delingette,et al.  Learning a Probabilistic Model for Diffeomorphic Registration , 2018, IEEE Transactions on Medical Imaging.

[50]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[51]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[52]  Mert R. Sabuncu,et al.  Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces , 2019, Medical Image Anal..

[53]  Calyampudi R. Rao.  Theory of Statistical Inference , 2008 .

[54]  Hua Yang,et al.  Deformable image registration of sliding organs using anisotropic diffusive regularization , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[55]  Quanzheng Li,et al.  Iterative PET Image Reconstruction Using Convolutional Neural Network Representation , 2017, IEEE Transactions on Medical Imaging.

[56]  Yu-Ping Wang,et al.  Scale-Space Derived From B-Splines , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Eric C Frey,et al.  A projection image database to investigate factors affecting image quality in weight-based dosing: application to pediatric renal SPECT , 2018, Physics in medicine and biology.

[58]  Michael S. Beauchamp,et al.  A new method for improving functional-to-structural MRI alignment using local Pearson correlation , 2009, NeuroImage.

[59]  Kouichi Sakurai,et al.  One Pixel Attack for Fooling Deep Neural Networks , 2017, IEEE Transactions on Evolutionary Computation.

[60]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[61]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[62]  Stephen M. Moore,et al.  The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository , 2013, Journal of Digital Imaging.

[63]  W. Segars,et al.  4D XCAT phantom for multimodality imaging research. , 2010, Medical physics.

[64]  Chih-Chieh Liu,et al.  PET Image Denoising Using a Deep Neural Network Through Fine Tuning , 2019, IEEE Transactions on Radiation and Plasma Medical Sciences.

[65]  Qian Wang,et al.  Deformable Image Registration Based on Similarity-Steered CNN Regression , 2017, MICCAI.

[66]  Brian B. Avants,et al.  Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain , 2008, Medical Image Anal..

[67]  Katsuyuki Taguchi,et al.  False dyssynchrony: problem with image-based cardiac functional analysis using x-ray computed tomography , 2017, Medical Imaging.

[68]  W P Segars,et al.  Population of anatomically variable 4D XCAT adult phantoms for imaging research and optimization. , 2013, Medical physics.