Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT

Mathematical model observers (MOs) have become popular in task-based CT image quality assessment, since, once proven to be correlated with human observers (HOs), these MOs can be used to estimate HO performance. However, typical MO studies are limited to phantom data which only involve uniform background. In practice, anatomical background variability and tissue non-uniformity affect HO lesion detection performance. Recently, we have proposed a deep-learning-based MO (DL-MO). In this study, we aim to investigate the correlation between this DL-MO and HOs for a lung-nodule localization task in chest CT. Using a patient database that contains 50 lung cancer screening CT patient cases, 12 different experimental conditions were generated, including 4 radiation dose levels, 3 nodule sizes, 2 nodule types and 3 reconstruction types. These conditions were created by using a validated noise and lesion insertion tool. Four subspecialized radiologists performed the HO study for all 12 conditions individually in a randomized fashion. The DL-MO was trained and tested for the same dataset. The performance of DL-MO and HO was compared across all the experimental conditions. DL-MO performance was strongly correlated with HO performance (Pearson’s correlation coefficient: 0.988 with a 95% confidence interval of [0.894, 0.999]). These results demonstrate the potential to use the proposed DL-MO to predict HO performance for the task of lung nodule localization in chest CT.

[1]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Premkumar Elangovan,et al.  A deep learning model observer for use in alterative forced choice virtual clinical trials , 2018, Medical Imaging.

[4]  Qiyuan Hu,et al.  A virtual clinical trial using projection-based nodule insertion to determine radiologist reader performance in lung cancer screening CT , 2017, Medical Imaging.

[5]  Felix K. Kopp,et al.  CNN as model observer in a liver lesion detection task for x‐ray computed tomography: A phantom study , 2018, Medical physics.

[6]  M. Shiung,et al.  Development and Validation of a Practical Lower-Dose-Simulation Tool for Optimizing Computed Tomography Scan Protocols , 2012, Journal of computer assisted tomography.

[7]  Shuai Leng,et al.  Lesion insertion in projection domain for computed tomography image quality assessment , 2015, Medical Imaging.

[8]  Shuai Leng,et al.  Prediction of human observer performance in a 2-alternative forced choice low-contrast detection task using channelized Hotelling observer: impact of radiation dose and reconstruction algorithms. , 2013, Medical physics.

[9]  Matthew A Kupinski,et al.  Correlation between a 2D channelized Hotelling observer and human observers in a low‐contrast detection task with multislice reading in CT , 2017, Medical physics.

[10]  Shuai Leng,et al.  Correlation between model observer and human observer performance in CT imaging when lesion location is uncertain , 2012, Medical Imaging.

[11]  Francesc Massanes,et al.  Evaluation of CNN as anthropomorphic model observer , 2017, Medical Imaging.