NeSVoR: Implicit Neural Representation for Slice-to-Volume Reconstruction in MRI

Reconstructing 3D MR volumes from multiple motion-corrupted stacks of 2D slices has shown promise in imaging of moving subjects, e. g., fetal MRI. However, existing slice-to-volume reconstruction methods are time-consuming, especially when a high-resolution volume is desired. Moreover, they are still vulnerable to severe subject motion and when image artifacts are present in acquired slices. In this work, we present NeSVoR, a resolution-agnostic slice-to-volume reconstruction method, which models the underlying volume as a continuous function of spatial coordinates with implicit neural representation. To improve robustness to subject motion and other image artifacts, we adopt a continuous and comprehensive slice acquisition model that takes into account rigid inter-slice motion, point spread function, and bias fields. NeSVoR also estimates pixel-wise and slice-wise variances of image noise and enables removal of outliers during reconstruction and visualization of uncertainty. Extensive experiments are performed on both simulated and in vivo data to evaluate the proposed method. Results show that NeSVoR achieves state-of-the-art reconstruction quality while providing two to ten-fold acceleration in reconstruction times over the state-of-the-art algorithms.

[1]  J. E. Iglesias,et al.  SVoRT: Iterative Transformer for Slice-to-Volume Registration in Fetal Brain MRI , 2022, MICCAI.

[2]  Yi Zhang,et al.  AFFIRM: Affinity Fusion-Based Framework for Iteratively Random Motion Correction of Multi-Slice Fetal Brain MRI , 2022, IEEE Transactions on Medical Imaging.

[3]  T. Müller,et al.  Instant neural graphics primitives with a multiresolution hash encoding , 2022, ACM Trans. Graph..

[4]  P. Golland,et al.  Automated detection and reacquisition of motion‐degraded images in fetal HASTE imaging at 3 T , 2021, Magnetic resonance in medicine.

[5]  Ana I. L. Namburete,et al.  ImplicitVol: Sensorless 3D Ultrasound Reconstruction with Deep Implicit Representation , 2021, ArXiv.

[6]  Jingyi Yu,et al.  IREM: High-Resolution Magnetic Resonance (MR) Image Reconstruction via Implicit Neural Representation , 2021, MICCAI.

[7]  Thomas Müller,et al.  Real-time neural radiance caching for path tracing , 2021, ACM Trans. Graph..

[8]  P. Ellen Grant,et al.  STRESS: Super-Resolution for Dynamic Fetal MRI using Self-Supervised Learning , 2021, MICCAI.

[9]  Marc Modat,et al.  MONAIfbs: MONAI-based fetal brain MRI deep learning segmentation , 2021, ArXiv.

[10]  A. Kouwe,et al.  Rapid head‐pose detection for automated slice prescription of fetal‐brain MRI , 2021, Int. J. Imaging Syst. Technol..

[11]  V. Prisacariu,et al.  NeRF-: Neural Radiance Fields Without Known Camera Parameters , 2021, ArXiv.

[12]  Hamza Kebiri,et al.  An automatic multi-tissue human fetal brain segmentation benchmark using the Fetal Tissue Annotation Dataset , 2020, Scientific Data.

[13]  Kai Zhang,et al.  NeRF++: Analyzing and Improving Neural Radiance Fields , 2020, ArXiv.

[14]  Jonathan T. Barron,et al.  NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Borjan Gagoski,et al.  Semi-Supervised Learning for Fetal Brain MRI Quality Assessment with ROI consistency , 2020, MICCAI.

[16]  Jonathan T. Barron,et al.  Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains , 2020, NeurIPS.

[17]  Richard A. Newcombe,et al.  Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction , 2020, ECCV.

[18]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[19]  S. Ourselin,et al.  TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning , 2020, Comput. Methods Programs Biomed..

[20]  Sébastien Ourselin,et al.  An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain MRI , 2019, NeuroImage.

[21]  P. Ellen Grant,et al.  Fetal Pose Estimation in Volumetric MRI using a 3D Convolution Neural Network , 2019, MICCAI.

[22]  Laurence H. Jackson,et al.  Deformable Slice-to-Volume Registration for Motion Correction of Fetal Body and Placenta MRI , 2019, IEEE Transactions on Medical Imaging.

[23]  Gordon Wetzstein,et al.  Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations , 2019, NeurIPS.

[24]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[25]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[26]  Hao Wu,et al.  Mixed Precision Training , 2017, ICLR.

[27]  Bishesh Khanal,et al.  3-D Reconstruction in Canonical Co-Ordinate Space From Arbitrarily Oriented 2-D Images , 2017, IEEE Transactions on Medical Imaging.

[28]  David Lopez-Paz,et al.  Optimizing the Latent Space of Generative Networks , 2017, ICML.

[29]  Amir Alansary,et al.  PVR: Patch-to-Volume Reconstruction for Large Area Motion Correction of Fetal MRI , 2016, IEEE Transactions on Medical Imaging.

[30]  Tomoki Arichi,et al.  A dedicated neonatal brain imaging system , 2016, Magnetic resonance in medicine.

[31]  Simon K. Warfield,et al.  Motion-Robust Diffusion-Weighted Brain MRI Reconstruction Through Slice-Level Registration-Based Motion Tracking , 2016, IEEE Transactions on Medical Imaging.

[32]  Freddy Odille,et al.  Motion-Corrected, Super-Resolution Reconstruction for High-Resolution 3D Cardiac Cine MRI , 2015, MICCAI.

[33]  Xavier Bresson,et al.  An efficient total variation algorithm for super-resolution in fetal brain MRI with adaptive regularization , 2015, NeuroImage.

[34]  Daniel Rueckert,et al.  Fast Volume Reconstruction from Motion Corrupted Stacks of 2D Slices , 2015, IEEE Transactions on Medical Imaging.

[35]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[36]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[37]  S. Saleem,et al.  Fetal MRI: An approach to practice: A review , 2013, Journal of advanced research.

[38]  Mary A. Rutherford,et al.  Reconstruction of fetal brain MRI with intensity matching and complete outlier removal , 2012, Medical Image Anal..

[39]  Simon K. Warfield,et al.  Robust Super-Resolution Volume Reconstruction From Slice Acquisitions: Application to Fetal Brain MRI , 2010, IEEE Transactions on Medical Imaging.

[40]  Daniel Rueckert,et al.  MRI of Moving Subjects Using Multislice Snapshot Images With Volume Reconstruction (SVR): Application to Fetal, Neonatal, and Adult Brain Studies , 2007, IEEE Transactions on Medical Imaging.

[41]  Colin Studholme,et al.  Registration-based approach for reconstruction of high-resolution in utero fetal MR brain images. , 2006, Academic radiology.

[42]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[43]  David H. Laidlaw,et al.  Partial-volume Bayesian classification of material mixtures in MR volume data using voxel histograms , 1997, IEEE Transactions on Medical Imaging.

[44]  H. Gudbjartsson,et al.  The rician distribution of noisy mri data , 1995, Magnetic resonance in medicine.

[45]  Colin Studholme,et al.  Intersection Based Motion Correction of Multislice MRI for 3-D in Utero Fetal Brain Image Formation , 2010, IEEE Transactions on Medical Imaging.