Real-time tumor localization with single x-ray projection at arbitrary gantry angles using a convolutional neural network (CNN)

PURPOSE For tumor tracking therapy, precise knowledge of tumor position in real-time is very important. A technique using single x-ray projection based on convolutional neural network (CNN) was recently developed which can achieve accurate tumor localization in real-time. However this method was only validated at fixed gantry angles. In this study, an improved technique is developed to handle arbitrary gantry angles for rotational radiotherapy. METHODS AND MATERIALS To evaluate the highly complex relationship between x-ray projections at arbitrary angles and tumor motion, a special CNN was proposed. In this network, a binary region of interest mask was applied on every extracted feature map. This avoids the overfitting problem due to gantry rotation by directing the network to neglect those irrelevant pixels whose intensity variation had nothing to do with breathing motion. In addition, an angle-dependent fully connection layer was utilized to recover the mapping from extracted feature maps to tumor motion, which would vary with the gantry angles. The method was tested with images from 15 realistic patients and compared with a variant network of VGG. RESULTS The tumors were clearly visible on x-ray projections for five patients only. The average tumor localization error was under 1.8 mm and 1.0 mm in superior-inferior and lateral directions. For the other ten patients whose tumors were no clearly visible in the x-ray projection, a feature point localization error was computed to evaluate the proposed method, whose mean value were no more than 1.5 mm and 1.0 mm in both directions for all patients. CONCLUSION A tumor localization method for single x-ray projection at arbitrary angle based on novel CNN was developed and validated in this study for real time operation. This greatly expanded the applicability of the tumor localization framework to the rotation therapy.

[1]  Paul J Keall,et al.  Retrospective analysis of artifacts in four-dimensional CT images of 50 abdominal and thoracic radiotherapy patients. , 2008, International journal of radiation oncology, biology, physics.

[2]  Mitsuhiro Nakamura,et al.  The accuracy of extracted target motion trajectories in four-dimensional cone-beam computed tomography for lung cancer patients. , 2016, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[3]  Fang-Fang Yin,et al.  A technique for estimating 4D-CBCT using prior knowledge and limited-angle projections. , 2013, Medical physics.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Nicholas Ayache,et al.  Symmetric Log-Domain Diffeomorphic Registration: A Demons-Based Approach , 2008, MICCAI.

[6]  Sasa Mutic,et al.  Clinical evaluations of an amplitude-based binning algorithm for 4DCT reconstruction in radiation therapy. , 2012, Medical physics.

[7]  Makoto Yoshizawa,et al.  Markerless Lung Tumor Motion Tracking by Dynamic Decomposition of X-Ray Image Intensity , 2013, Journal of medical engineering.

[8]  Qinghui Zhang,et al.  A patient-specific respiratory model of anatomical motion for radiation treatment planning. , 2007, Medical physics.

[9]  D Ruan,et al.  Real-time prediction of respiratory motion based on local regression methods , 2007, Physics in medicine and biology.

[10]  Jan-Jakob Sonke,et al.  4D CT amplitude binning for the generation of a time-averaged 3D mid-position CT scan , 2014, Physics in medicine and biology.

[11]  Seonyeong Park,et al.  A Novel Method of Cone Beam CT Projection Binning Based on Image Registration , 2017, IEEE Transactions on Medical Imaging.

[12]  D. Low,et al.  A comparison between amplitude sorting and phase-angle sorting using external respiratory measurement for 4D CT. , 2006, Medical physics.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Shinichi Shimizu,et al.  A motion-compensated image filter for low-dose fluoroscopy in a real-time tumor-tracking radiotherapy system , 2014, Journal of radiation research.

[15]  Steve B. Jiang,et al.  Markerless lung tumor tracking and trajectory reconstruction using rotational cone-beam projections: a feasibility study , 2010, Physics in medicine and biology.

[16]  Steve B. Jiang,et al.  Fluoroscopic tracking of multiple implanted fiducial markers using multiple object tracking , 2007, Physics in medicine and biology.

[17]  Kumaradevan Punithakumar,et al.  A Fully Convolutional Deep Neural Network for Lung Tumor Boundary Tracking in MRI , 2018, 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[18]  Xiangzhi Bai,et al.  Convolutional Neural Network (CNN) Based Three Dimensional Tumor Localization Using Single X-Ray Projection , 2019, IEEE Access.

[19]  Zdenka Kuncic,et al.  Markerless tumor tracking using short kilovoltage imaging arcs for lung image-guided radiotherapy , 2015, Physics in medicine and biology.

[20]  Steve B. Jiang,et al.  Fluoroscopic tumor tracking for image-guided lung cancer radiotherapy , 2009, Physics in medicine and biology.

[21]  Yuichi Motai,et al.  Respiratory Motion Estimation With Hybrid Implementation of Extended Kalman Filter , 2012, IEEE Transactions on Industrial Electronics.

[22]  Jerry R. Van Aken An Efficient Ellipse-Drawing Algorithm , 1984, IEEE Computer Graphics and Applications.

[23]  A method for volumetric imaging in radiotherapy using single x-ray projection. , 2015, Medical physics.

[24]  Gregory C Sharp,et al.  Prediction of respiratory tumour motion for real-time image-guided radiotherapy. , 2004, Physics in medicine and biology.

[25]  Steve B. Jiang,et al.  Markerless gating for lung cancer radiotherapy based on machine learning techniques , 2009, Physics in medicine and biology.

[26]  Rakesh Patel,et al.  Markerless motion tracking of lung tumors using dual-energy fluoroscopy. , 2014, Medical physics.

[27]  Tinsu Pan,et al.  Phase and amplitude binning for 4D-CT imaging. , 2007 .

[28]  Xun Jia,et al.  3D tumor localization through real-time volumetric x-ray imaging for lung cancer radiotherapy. , 2011, Medical physics.