Convolutional Neural Networks with Template-Based Data Augmentation for Functional Lung Image Quantification.

RATIONALE AND OBJECTIVES We propose an automated segmentation pipeline based on deep learning for proton lung MRI segmentation and ventilation-based quantification which improves on our previously reported methodologies in terms of computational efficiency while demonstrating accuracy and robustness. The large data requirement for the proposed framework is made possible by a novel template-based data augmentation strategy. Supporting this work is the open-source ANTsRNet-a growing repository of well-known deep learning architectures first introduced here. MATERIALS AND METHODS Deep convolutional neural network (CNN) models were constructed and trained using a custom multilabel Dice metric loss function and a novel template-based data augmentation strategy. Training (including template generation and data augmentation) employed 205 proton MR images and 73 functional lung MRI. Evaluation was performed using data sets of size 63 and 40 images, respectively. RESULTS Accuracy for CNN-based proton lung MRI segmentation (in terms of Dice overlap) was left lung: 0.93 ± 0.03, right lung: 0.94 ± 0.02, and whole lung: 0.94 ± 0.02. Although slightly less accurate than our previously reported joint label fusion approach (left lung: 0.95 ± 0.02, right lung: 0.96 ± 0.01, and whole lung: 0.96 ± 0.01), processing time is <1 second per subject for the proposed approach versus ∼30 minutes per subject using joint label fusion. Accuracy for quantifying ventilation defects was determined based on a consensus labeling where average accuracy (Dice multilabel overlap of ventilation defect regions plus normal region) was 0.94 for the CNN method; 0.92 for our previously reported method; and 0.90, 0.92, and 0.94 for expert readers. CONCLUSION The proposed framework yields accurate automated quantification in near real time. CNNs drastically reduce processing time after offline model construction and demonstrate significant future potential for facilitating quantitative analysis of functional lung MRI.

[1]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[3]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Paul A. Yushkevich,et al.  Multi-Atlas Segmentation with Joint Label Fusion , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Brian B. Avants,et al.  Explicit B-spline regularization in diffeomorphic image registration , 2013, Front. Neuroinform..

[7]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[8]  John P Mugler,et al.  Use of hyperpolarized helium-3 MRI to assess response to ivacaftor treatment in patients with cystic fibrosis. , 2017, Journal of cystic fibrosis : official journal of the European Cystic Fibrosis Society.

[9]  Rohan S. Virgincar,et al.  Extending semiautomatic ventilation defect analysis for hyperpolarized (129)Xe ventilation MRI. , 2014, Academic radiology.

[10]  John P Mugler,et al.  Clinical correlates of lung ventilation defects in asthmatic children. , 2016, The Journal of allergy and clinical immunology.

[11]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Sean B Fain,et al.  Magnetic resonance imaging with hyperpolarized agents: methods and applications , 2017, Physics in medicine and biology.

[14]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[15]  Bastiaan Driehuys,et al.  Hyperpolarized Gas MR Imaging: Technique and Applications. , 2015, Magnetic resonance imaging clinics of North America.

[16]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[17]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[18]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[19]  Brian B. Avants,et al.  The optimal template effect in hippocampus studies of diseased populations , 2010, NeuroImage.

[20]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[21]  Geoff S. Nitschke,et al.  Improving Deep Learning with Generic Data Augmentation , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[22]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[23]  Jim M. Wild,et al.  Spatial fuzzy c‐means thresholding for semiautomated calculation of percentage lung ventilated volume from hyperpolarized gas and 1H MRI , 2018, Journal of magnetic resonance imaging : JMRI.

[24]  Hiroto Hatabu,et al.  Pulmonary hyperpolarized noble gas MRI: recent advances and perspectives in clinical application. , 2014, European journal of radiology.

[25]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  N. Tustison,et al.  Atlas‐based estimation of lung and lobar anatomy in proton MRI , 2016, Magnetic resonance in medicine.

[29]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[30]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[31]  William M. Wells,et al.  Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation , 2004, IEEE Transactions on Medical Imaging.

[32]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[33]  Mario Castro,et al.  Using imaging as a biomarker for asthma , 2017, The Journal of allergy and clinical immunology.

[34]  Arno Klein,et al.  A reproducible evaluation of ANTs similarity metric performance in brain image registration , 2011, NeuroImage.

[35]  Brian B. Avants,et al.  Convolutional Neural Networks for Rapid and Simultaneous Brain Extraction and Tissue Segmentation , 2018 .

[36]  Mark D. McDonnell,et al.  Understanding Data Augmentation for Classification: When to Warp? , 2016, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA).

[37]  Sean B Fain,et al.  Semiautomated Ventilation Defect Quantification in Exercise-induced Bronchoconstriction Using Hyperpolarized Helium-3 Magnetic Resonance Imaging: A Repeatability Study. , 2016, Academic radiology.

[38]  J. Gee,et al.  Ventilation‐based segmentation of the lungs using hyperpolarized 3He MRI , 2011, Journal of magnetic resonance imaging : JMRI.

[39]  John P Mugler,et al.  Feature analysis of hyperpolarized helium‐3 pulmonary MRI: A study of asthmatics versus nonasthmatics , 2010, Magnetic resonance in medicine.

[40]  Pierrick Coupé,et al.  Author manuscript, published in "Journal of Magnetic Resonance Imaging 2010;31(1):192-203" DOI: 10.1002/jmri.22003 Adaptive Non-Local Means Denoising of MR Images with Spatially Varying Noise Levels , 2010 .

[41]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Miranda Kirby,et al.  Hyperpolarized 3He magnetic resonance functional imaging semiautomated segmentation. , 2012, Academic radiology.

[43]  G Wilson Miller,et al.  Rapid acquisition of helium‐3 and proton three‐dimensional image sets of the human lung in a single breath‐hold using compressed sensing , 2015, Magnetic resonance in medicine.

[44]  Miranda Kirby,et al.  What are ventilation defects in asthma? , 2013, Thorax.

[45]  Miranda Kirby,et al.  Hyperpolarized (3)He ventilation defects used to predict pulmonary exacerbations in mild to moderate chronic obstructive pulmonary disease. , 2014, Radiology.