Deep architecture neural network-based real-time image processing for image-guided radiotherapy.

INTRODUCTION To develop real-time image processing for image-guided radiotherapy, we evaluated several neural network models for use with different imaging modalities, including X-ray fluoroscopic image denoising. METHODS & MATERIALS Setup images of prostate cancer patients were acquired with two oblique X-ray fluoroscopic units. Two types of residual network were designed: a convolutional autoencoder (rCAE) and a convolutional neural network (rCNN). We changed the convolutional kernel size and number of convolutional layers for both networks, and the number of pooling and upsampling layers for rCAE. The ground-truth image was applied to the contrast-limited adaptive histogram equalization (CLAHE) method of image processing. Network models were trained to keep the quality of the output image close to that of the ground-truth image from the input image without image processing. For image denoising evaluation, noisy input images were used for the training. RESULTS More than 6 convolutional layers with convolutional kernels >5×5 improved image quality. However, this did not allow real-time imaging. After applying a pair of pooling and upsampling layers to both networks, rCAEs with >3 convolutions each and rCNNs with >12 convolutions with a pair of pooling and upsampling layers achieved real-time processing at 30 frames per second (fps) with acceptable image quality. CONCLUSIONS Use of our suggested network achieved real-time image processing for contrast enhancement and image denoising by the use of a conventional modern personal computer.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Chengjin Zhang,et al.  Denoising Convolutional Neural Network , 2015, 2015 IEEE International Conference on Information and Automation.

[3]  Katsuyuki Tanimoto,et al.  Carbon-Ion Pencil Beam Scanning Treatment With Gated Markerless Tumor Tracking: An Analysis of Positional Accuracy. , 2016, International journal of radiation oncology, biology, physics.

[4]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[5]  P C Levendag,et al.  Lung tumor tracking during stereotactic radiotherapy treatment with the CyberKnife: Marker placement and early results , 2006, Acta oncologica.

[6]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[7]  Rie Tanaka,et al.  Improved accuracy of markerless motion tracking on bone suppression images: preliminary study for image-guided radiation therapy (IGRT) , 2015, Physics in medicine and biology.

[8]  Rakesh Patel,et al.  Markerless motion tracking of lung tumors using dual-energy fluoroscopy. , 2014, Medical physics.

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Shinichi Shimizu,et al.  Real-time tumour-tracking radiotherapy , 1999, The Lancet.

[11]  S. Mori,et al.  Current status and future prospects of multi-dimensional image-guided particle therapy , 2013, Radiological Physics and Technology.

[12]  Enhong Chen,et al.  Image Denoising and Inpainting with Deep Neural Networks , 2012, NIPS.

[13]  Makoto Yoshizawa,et al.  A kernel-based method for markerless tumor tracking in kV fluoroscopic images , 2014, Physics in medicine and biology.

[14]  Arthur L Boyer,et al.  Respiratory gated beam delivery cannot facilitate margin reduction, unless combined with respiratory correlated image guidance. , 2008, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[15]  Takuji Furukawa,et al.  Patient handling system for carbon ion beam scanning therapy , 2012, Journal of applied clinical medical physics.

[16]  H. Sebastian Seung,et al.  Natural Image Denoising with Convolutional Networks , 2008, NIPS.

[17]  Qianjin Feng,et al.  Cascade of multi-scale convolutional neural networks for bone suppression of chest radiographs in gradient domain , 2017, Medical Image Anal..

[18]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[19]  Hideaki Haneishi,et al.  Development of fast patient position verification software using 2D-3D image registration and its clinical experience , 2015, Journal of radiation research.

[20]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[21]  Steve B. Jiang,et al.  Fluoroscopic tumor tracking for image-guided lung cancer radiotherapy , 2009, Physics in medicine and biology.

[22]  Fang-Fang Yin,et al.  Use of the BrainLAB ExacTrac X-Ray 6D system in image-guided radiotherapy. , 2008, Medical dosimetry : official journal of the American Association of Medical Dosimetrists.

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  Steve B. Jiang,et al.  Multiple template-based fluoroscopic tracking of lung tumor mass without implanted fiducial markers , 2007, Physics in medicine and biology.

[25]  Gregory C Sharp,et al.  A review of image-guided radiotherapy , 2009, Radiological physics and technology.

[26]  Fang-Fang Yin,et al.  6D image guidance for spinal non-invasive stereotactic body radiation therapy: Comparison between ExacTrac X-ray 6D with kilo-voltage cone-beam CT. , 2010, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[27]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.