Ultra-short echo-time magnetic resonance imaging lung segmentation with under-Annotations and domain shift

Ultra-short echo-time (UTE) magnetic resonance imaging (MRI) provides enhanced visualization of pulmonary structural and functional abnormalities and has shown promise in phenotyping lung disease. Here, we describe the development and evaluation of a lung segmentation approach to facilitate UTE MRI methods for patient-based imaging. The proposed approach employs a k-means algorithm in kernel space for pair-wise feature clustering and imposes image domain continuous regularization, coined as continuous kernel k-means (CKKM). The high-order CKKM algorithm was simplified through upper bound relaxation and solved within an iterative continuous max-flow framework. We combined the CKKM with U-net and atlas-based approaches and comprehensively evaluated the performance on 100 images from 25 patients with asthma and bronchial pulmonary dysplasia enrolled at Robarts Research Institute (Western University, London, Canada) and Centre Hospitalier Universitaire (Sainte-Justine, Montreal, Canada). For U-net, we trained the network five times on a mixture of five different images with under-annotations and applied the model to 64 images from the two centres. We also trained a U-net on five images with full and brush annotations from one centre, and tested the model on 32 images from the other centre. For an atlas-based approach, we employed three atlas images to segment 64 target images from the two centres through straightforward atlas registration and label fusion. We applied the CKKM algorithm to the baseline U-net and atlas outputs and refined the initial segmentation through multi-volume image fusion. The integration of CKKM substantially improved baseline results and yielded, with minimal computational cost, segmentation accuracy, and precision that were greater than some state-of-the-art deep learning models and similar to experienced observer manual segmentation. This suggests that deep learning and atlas-based approaches may be utilized to segment UTE MRI datasets using relatively small training datasets with under-annotations.

[1]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[2]  Nima Tajbakhsh,et al.  UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[3]  Jean A. Tkach,et al.  Quantification of neonatal lung parenchymal density via ultrashort echo time MRI with comparison to CT , 2017, Journal of magnetic resonance imaging : JMRI.

[4]  C J Bergin,et al.  Magnetic resonance imaging of lung parenchyma , 1993, Journal of thoracic imaging.

[5]  Shunxing Bao,et al.  SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth , 2018, IEEE Transactions on Medical Imaging.

[6]  Andrea Aliverti,et al.  Assessment of pulmonary structure–function relationships in young children and adolescents with cystic fibrosis by multivolume proton‐MRI and CT , 2018, Journal of magnetic resonance imaging : JMRI.

[7]  Mitko Veta,et al.  Intensity Augmentation to Improve Generalizability of Breast Segmentation Across Different MRI Scan Protocols , 2020, IEEE Transactions on Biomedical Engineering.

[8]  Ismail Ben Ayed,et al.  Kernel Cuts: Kernel and Spectral Clustering Meet Regularization , 2018, International Journal of Computer Vision.

[9]  Zhipeng Jia,et al.  Constrained Deep Weak Supervision for Histopathology Image Segmentation , 2017, IEEE Transactions on Medical Imaging.

[10]  N. Müller,et al.  "Density mask". An objective method to quantitate emphysema using computed tomography. , 1988, Chest.

[11]  Ismail Ben Ayed,et al.  On Regularized Losses for Weakly-supervised CNN Segmentation , 2018, ECCV.

[12]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[13]  Jae Y. Shin,et al.  Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? , 2016, IEEE transactions on medical imaging.

[14]  N. Tustison,et al.  Atlas‐based estimation of lung and lobar anatomy in proton MRI , 2016, Magnetic resonance in medicine.

[15]  Xue-Cheng Tai,et al.  A study on continuous max-flow and min-cut approaches , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Jie Li,et al.  A sparse annotation strategy based on attention-guided active learning for 3D medical image segmentation , 2019, ArXiv.

[17]  Lena Gorelick,et al.  Auxiliary Cuts for General Classes of Higher Order Functionals , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Ahmed Hosny,et al.  Artificial intelligence in radiology , 2018, Nature Reviews Cancer.

[19]  Edwin K Silverman,et al.  CT-Definable Subtypes of Chronic Obstructive Pulmonary Disease: A Statement of the Fleischner Society. , 2015, Radiology.

[20]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[21]  Brian B. Avants,et al.  N4ITK: Improved N3 Bias Correction , 2010, IEEE Transactions on Medical Imaging.

[22]  Daguang Xu,et al.  Generalizing Deep Learning for Medical Image Segmentation to Unseen Domains via Deep Stacked Transformation , 2020, IEEE Transactions on Medical Imaging.

[23]  Nima Tajbakhsh,et al.  Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation , 2019, Medical Image Anal..

[24]  Grace Parraga,et al.  Ultra‐short echo‐time pulmonary MRI: Evaluation and reproducibility in COPD subjects with and without bronchiectasis , 2015, Journal of magnetic resonance imaging : JMRI.

[25]  Ender Konukoglu,et al.  Learning to Segment Medical Images with Scribble-Supervision Alone , 2018, DLMIA/ML-CDS@MICCAI.

[26]  Michael Deimling,et al.  Non‐contrast‐enhanced perfusion and ventilation assessment of the human lung by means of fourier decomposition in proton MRI , 2009, Magnetic resonance in medicine.

[27]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[28]  Michel Montaudon,et al.  Lung morphology assessment of cystic fibrosis using MRI with ultra-short echo time at submillimeter spatial resolution , 2016, European Radiology.

[29]  Aaron Fenster,et al.  Globally optimal co-segmentation of three-dimensional pulmonary 1H and hyperpolarized 3He MRI with spatial consistence prior , 2015, Medical Image Anal..

[30]  T. Vercauteren,et al.  Scribble-based Domain Adaptation via Co-segmentation , 2020, MICCAI.

[31]  Miranda Kirby,et al.  Pulmonary Imaging Biomarkers of Gas Trapping and Emphysema in COPD: (3)He MR Imaging and CT Parametric Response Maps. , 2016, Radiology.

[32]  D. Brenner,et al.  Estimated risks of radiation-induced fatal cancer from pediatric CT. , 2001, AJR. American journal of roentgenology.

[33]  Thomas Brox,et al.  U-Net: deep learning for cell counting, detection, and morphometry , 2018, Nature Methods.

[34]  Kevin M. Johnson,et al.  Optimized 3D ultrashort echo time pulmonary MRI , 2013, Magnetic resonance in medicine.

[35]  Alexei A. Efros,et al.  Unbiased look at dataset bias , 2011, CVPR 2011.

[36]  A Fenster,et al.  Anatomical pulmonary magnetic resonance imaging segmentation for regional structure-function measurements of asthma. , 2016, Medical physics.

[37]  Grace Parraga,et al.  Ultrashort echo time MRI biomarkers of asthma , 2017, Journal of magnetic resonance imaging : JMRI.

[38]  W. Happer,et al.  Biological magnetic resonance imaging using laser-polarized 129Xe , 1994, Nature.

[39]  Ismail Ben Ayed,et al.  Secrets of GrabCut and Kernel K-Means , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[40]  Cynthia Rudin,et al.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , 2018, Nature Machine Intelligence.

[41]  Robert R. Edelman,et al.  Noninvasive assessment of regional ventilation in the human lung using oxygen–enhanced magnetic resonance imaging , 1996, Nature Medicine.

[42]  Eric Granger,et al.  Constrained‐CNN losses for weakly supervised segmentation☆ , 2018, Medical Image Anal..

[43]  I. Ball,et al.  Inert fluorinated gas MRI: a new pulmonary imaging modality , 2014, NMR in biomedicine.

[44]  Elsa D. Angelini,et al.  Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules , 2017, MICCAI.

[45]  R. Szczesniak,et al.  Ultrashort Echo-Time Magnetic Resonance Imaging Is a Sensitive Method for the Evaluation of Early Cystic Fibrosis Lung Disease. , 2016, Annals of the American Thoracic Society.

[46]  Thomas Brox,et al.  3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation , 2016, MICCAI.

[47]  N L Müller,et al.  MR imaging of the lungs: value of short TE spin-echo pulse sequences. , 1992, AJR. American journal of roentgenology.

[48]  Y. Crémillieux,et al.  Longitudinal and noninvasive assessment of emphysema evolution in a murine model using proton MRI , 2012, Magnetic resonance in medicine.

[49]  William Parker,et al.  A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[50]  Grace Parraga,et al.  Using pulmonary imaging to move chronic obstructive pulmonary disease beyond FEV1. , 2014, American journal of respiratory and critical care medicine.

[51]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[52]  Edwin K Silverman,et al.  Quantitative Computed Tomography of the Lungs and Airways in Healthy Nonsmoking Adults , 2012, Investigative radiology.

[53]  Mert R. Sabuncu,et al.  Multi-atlas segmentation of biomedical images: A survey , 2014, Medical Image Anal..

[54]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[55]  Stefan Neubauer,et al.  Improving cardiac MRI convolutional neural network segmentation on small training datasets and dataset shift: A continuous kernel cut approach , 2020, Medical Image Anal..

[56]  Yoshiharu Ohno,et al.  Ultra‐short echo time (UTE) MR imaging of the lung: Comparison between normal and emphysematous lungs in mutant mice , 2010, Journal of magnetic resonance imaging : JMRI.

[57]  Daniel Rueckert,et al.  Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction , 2019, MICCAI.

[58]  Yuri Boykov,et al.  Normalized Cut Loss for Weakly-Supervised CNN Segmentation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[59]  D. Lynch,et al.  Pulmonary CT and MRI phenotypes that help explain chronic pulmonary obstruction disease pathophysiology and outcomes , 2016, Journal of magnetic resonance imaging : JMRI.

[60]  K. Jeon,et al.  Comparison of lung imaging using three-dimensional ultrashort echo time and zero echo time sequences: preliminary study , 2018, European Radiology.

[61]  Jie Xu,et al.  The practical implementation of artificial intelligence technologies in medicine , 2019, Nature Medicine.

[62]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.