Motion tracking in 3D ultrasound imaging based on Markov-like deep-learning-based deformable registration

Ultrasound imaging during or prior to radiation therapy offers a great potential in terms of safety, cost and real time imaging capacity. However, this task is challenging for tumors of abdominal such as liver cancer due to respiratory motion. In this work, we proposed an unsupervised deep-learning-based method to track the respiratory motion for 3D ultrasound (US) liver imaging. A Markov-like network, which extract features from consistent 3D US frames, was utilized to estimate a sequence of deformation vector fields (DVFs) that register tracked frame with landmark to match the coming untracked frames without landmark. Then, the landmarks of the coming frames were tracked by moving landmark position of tracked frame to the coming frames based on DVFs. The Markov-like network aims to consider motion consistency between each two frames and is implemented via a generative adversarial network (GAN). The proposed method was evaluated on the MICCAI CLUST 2015 challenge dataset. A retrospective study was performed with a total of 8 sets of 3D US sequence, and each 3D sequence has 4-96 frames marked with landmarks. We used a leave-one-out experiment to test our tracking method. Quantitative evaluation was performed by calculating the tracking error between estimated landmarks and the ground truth landmarks on each frame. Results of the proposed method showed a mean tracking error of 2.01±1.06 mm for 3D liver US images. We proposed a deep learning-based approach for 3D US liver motion tracking. This method reduces the processing time for tracking respiratory motion significantly, which can reduce the delivery uncertainty.

[1]  Steve B. Jiang,et al.  The management of respiratory motion in radiation oncology report of AAPM Task Group 76. , 2006, Medical physics.

[2]  Yang Lei,et al.  Synthetic MRI-aided Head-and-Neck Organs-at-Risk Auto-Delineation for CBCT-guided Adaptive Radiotherapy. , 2020, 2010.04275.

[3]  Yang Lei,et al.  Multiparametric MRI-guided dose boost to dominant intraprostatic lesions in CT-based High-dose-rate prostate brachytherapy. , 2019, The British journal of radiology.

[4]  Xiangyang Tang,et al.  CBCT-based Synthetic CT Generation using Deep-attention CycleGAN for Pancreatic Adaptive Radiotherapy. , 2020, Medical physics.

[5]  Sibo Tian,et al.  A planning study of focal dose escalations to multiparametric MRI-defined dominant intraprostatic lesions in prostate proton radiation therapy. , 2020, British Journal of Radiology.

[6]  Yang Lei,et al.  Multimodal MRI Synthesis Using Unified Generative Adversarial Networks. , 2020, Medical physics.

[7]  Jeffrey C Bamber,et al.  4D ultrasound speckle tracking of intra-fraction prostate motion: a phantom-based comparison with x-ray fiducial tracking using CyberKnife , 2014, Physics in medicine and biology.

[8]  Feng-Li Lian,et al.  Ultrasound image-guided algorithms for tracking liver motion , 2012, 2012 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).

[9]  Chunxu Shen,et al.  Discriminative Correlation Filter Network for Robust Landmark Tracking in Ultrasound Guided Intervention , 2019, MICCAI.

[10]  Yang Lei,et al.  A review on medical imaging synthesis using deep learning and its clinical applications , 2020, Journal of applied clinical medical physics.

[11]  Yang Lei,et al.  Automatic Multi-Catheter Detection using Deeply Supervised Convolutional Neural Network in MRI-guided HDR Prostate Brachytherapy. , 2020, Medical physics.

[12]  Christine Tanner,et al.  Siamese Networks With Location Prior for Landmark Tracking in Liver Ultrasound Sequences , 2019, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019).

[13]  Peter Kazanzides,et al.  2D ultrasound imaging based intra-fraction respiratory motion tracking for abdominal radiation therapy using machine learning , 2019, Physics in medicine and biology.

[14]  Gustavo Carneiro,et al.  Siam-U-Net: encoder-decoder siamese network for knee cartilage tracking in ultrasound images , 2019, Medical Image Anal..

[15]  Peter Kazanzides,et al.  System Integration and In Vivo Testing of a Robot for Ultrasound Guidance and Monitoring During Radiotherapy , 2017, IEEE Transactions on Biomedical Engineering.

[16]  Kun Wang,et al.  Cascaded one-shot deformable convolutional neural networks: Developing a deep learning model for respiratory motion estimation in ultrasound sequences , 2020, Medical Image Anal..

[17]  Xiangyang Tang,et al.  CT-based Multi-organ Segmentation using a 3D Self-attention U-Net Network for Pancreatic Radiotherapy. , 2020, Medical physics.

[18]  O Somphone,et al.  The 2014 liver ultrasound tracking benchmark , 2015, Physics in medicine and biology.

[19]  Santanu Chaudhury,et al.  Ultrasound Image Segmentation: A Deeply Supervised Network With Attention to Boundaries , 2019, IEEE Transactions on Biomedical Engineering.

[20]  Yong Yin,et al.  Attention-aware fully convolutional neural network with convolutional long short-term memory network for ultrasound-based motion tracking. , 2019, Medical physics.

[21]  Yabo Fu,et al.  Deep Learning in Multi-organ Segmentation , 2020, ArXiv.

[22]  Yang Lei,et al.  Intensity non-uniformity correction in MR imaging using residual cycle generative adversarial network , 2020, Physics in medicine and biology.