TiAVox: Time-aware Attenuation Voxels for Sparse-view 4D DSA Reconstruction

Four-dimensional Digital Subtraction Angiography (4D DSA) plays a critical role in the diagnosis of many medical diseases, such as Arteriovenous Malformations (AVM) and Arteriovenous Fistulas (AVF). Despite its significant application value, the reconstruction of 4D DSA demands numerous views to effectively model the intricate vessels and radiocontrast flow, thereby implying a significant radiation dose. To address this high radiation issue, we propose a Time-aware Attenuation Voxel (TiAVox) approach for sparse-view 4D DSA reconstruction, which paves the way for high-quality 4D imaging. Additionally, 2D and 3D DSA imaging results can be generated from the reconstructed 4D DSA images. TiAVox introduces 4D attenuation voxel grids, which reflect attenuation properties from both spatial and temporal dimensions. It is optimized by minimizing discrepancies between the rendered images and sparse 2D DSA images. Without any neural network involved, TiAVox enjoys specific physical interpretability. The parameters of each learnable voxel represent the attenuation coefficients. We validated the TiAVox approach on both clinical and simulated datasets, achieving a 31.23 Peak Signal-to-Noise Ratio (PSNR) for novel view synthesis using only 30 views on the clinically sourced dataset, whereas traditional Feldkamp-Davis-Kress methods required 133 views. Similarly, with merely 10 views from the synthetic dataset, TiAVox yielded a PSNR of 34.32 for novel view synthesis and 41.40 for 3D reconstruction. We also executed ablation studies to corroborate the essential components of TiAVox. The code will be publically available.

[1]  Yuewen Ma,et al.  Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields , 2023, 2023 IEEE/CVF International Conference on Computer Vision (ICCV).

[2]  Kai Wang,et al.  Neural LerPlane Representations for Fast 4D Reconstruction of Deformable Tissues , 2023, MICCAI.

[3]  Zhenguo Li,et al.  ConsistentNeRF: Enhancing Neural Radiance Fields with 3D Consistency for Sparse View Synthesis , 2023, ArXiv.

[4]  R. Soundararajan,et al.  ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fields , 2023, SIGGRAPH.

[5]  X. Li,et al.  Learning Deep Intensity Field for Extremely Sparse-View CBCT Reconstruction , 2023, MICCAI.

[6]  Ji-Yeon Kim,et al.  Temporal Interpolation is all You Need for Dynamic Neural Radiance Fields , 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  B. Recht,et al.  K-Planes: Explicit Radiance Fields in Space, Time, and Appearance , 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Justin Johnson,et al.  HexPlane: A Fast Representation for Dynamic Scenes , 2023, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Zhiming Cui,et al.  SNAF: Sparse-view CBCT Reconstruction with Neural Attenuation Fields , 2022, ArXiv.

[10]  Hongwen Zhang,et al.  Tensor4D: Efficient Neural 4D Decomposition for High-Fidelity Dynamic Reconstruction and Rendering , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Andreas Geiger,et al.  NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields , 2022, IEEE Transactions on Visualization and Computer Graphics.

[12]  Chengbo Liu,et al.  Self-supervised learning enables 3D digital subtraction angiography reconstruction from ultra-sparse 2D projection views: A multicenter study , 2022, Cell reports. Medicine.

[13]  Hongdong Li,et al.  NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction , 2022, MICCAI.

[14]  Hongjiang Wei,et al.  Self-Supervised Coordinate Projection Network for Sparse-View Computed Tomography , 2022, IEEE Transactions on Computational Imaging.

[15]  C. Theobalt,et al.  Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction , 2022, ICLR.

[16]  C. Brune,et al.  Going Off-Grid: Continuous Implicit Neural Representations for 3D Vascular Modeling , 2022, STACOM@MICCAI.

[17]  Yan Wang,et al.  Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera , 2022, NeurIPS.

[18]  Qingxu Dou,et al.  Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery , 2022, MICCAI.

[19]  M. Nießner,et al.  Fast Dynamic Radiance Fields with Time-Aware Neural Voxels , 2022, SIGGRAPH Asia.

[20]  Andreas Geiger,et al.  TensoRF: Tensorial Radiance Fields , 2022, ECCV.

[21]  W. Heidrich,et al.  NeAT: Neural Adaptive Tomography , 2022, ACM Trans. Graph..

[22]  Benjamin Recht,et al.  Plenoxels: Radiance Fields without Neural Networks , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jonathan T. Barron,et al.  RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Hwann-Tzong Chen,et al.  Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  W. Heidrich,et al.  IntraTomo: Self-supervised Learning-based Tomography via Sinogram Synthesis and Prediction , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  J. Pauly,et al.  NeRP: Implicit Neural Representation Learning With Prior Embedding for Sparsely Sampled Image Reconstruction , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Francesc Moreno-Noguer,et al.  D-NeRF: Neural Radiance Fields for Dynamic Scenes , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  S. Schafer,et al.  4D-DSA: Development and Current Neurovascular Applications , 2020, American Journal of Neuroradiology.

[29]  Kyaw Zaw Lin,et al.  Neural Sparse Voxel Fields , 2020, NeurIPS.

[30]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[31]  Yaoqin Xie,et al.  A Sparse-View CT Reconstruction Method Based on Combination of DenseNet and Deconvolution , 2018, IEEE Transactions on Medical Imaging.

[32]  Uwe Kruger,et al.  3-D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning From a 2-D Trained Network , 2018, IEEE Transactions on Medical Imaging.

[33]  Jong Chul Ye,et al.  Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT , 2017, IEEE Transactions on Medical Imaging.

[34]  T Struffert,et al.  4D DSA for Dynamic Visualization of Cerebral Vasculature: A Single-Center Experience in 26 Cases , 2017, American Journal of Neuroradiology.

[35]  Bruce R. Rosen,et al.  Image reconstruction by domain-transform manifold learning , 2017, Nature.

[36]  Michael Unser,et al.  Deep Convolutional Neural Network for Inverse Problems in Imaging , 2016, IEEE Transactions on Image Processing.

[37]  S. Schafer,et al.  A Comparison of 4D DSA with 2D and 3D DSA in the Analysis of Normal Vascular Structures in a Canine Model , 2015, American Journal of Neuroradiology.

[38]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[39]  Sebastian Schafer,et al.  4D DSA a new technique for arteriovenous malformation evaluation: a feasibility study , 2015, Journal of NeuroInterventional Surgery.

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  L. Feldkamp,et al.  Practical cone-beam algorithm , 1984 .