Deep-learning model observer for a low-contrast hepatic metastases localization task in computed tomography.

PURPOSE Conventional model observers (MO) in CT are often limited to a uniform background or varying background that is random and can be modelled in an analytical form. It is unclear if these conventional MOs can be readily generalized to predict human observer performance in clinical CT tasks that involve realistic anatomical background. Deep-learning-based model observers (DL-MO) have recently been developed, but have not been validated for challenging low contrast diagnostic tasks in abdominal CT. We consequently sought to validate a DL-MO for a low-contrast hepatic metastases localization task. METHODS We adapted our recently-developed DL-MO framework for the liver metastases localization task. Our previously-validated projection-domain lesion- / noise-insertion techniques were used to synthesize realistic positive and low-dose abdominal CT exams, using the archived patient projection data. Ten experimental conditions were generated, which involved different lesion sizes / contrasts, radiation dose levels, and image reconstruction types. Each condition included 100 trials generated from a patient cohort of 7 cases. Each trial was presented as liver image patches (160×160×5 voxels). The DL-MO performance was calculated for each condition and was compared with human observer performance, which was obtained by 3 sub-specialized radiologists in an observer study. The performance of DL-MO and radiologists was gauged by the area under localization receiver-operating-characteristic curves. The generalization performance of the DL-MO was estimated with the repeated 2-fold cross-validation method over the same set of trials used in human observer study. A multi-slice Channelized Hoteling Observers (CHO) was compared with the DL-MO across the same experimental conditions. RESULTS The performance of DL-MO was highly correlated to that of radiologists (Pearson's correlation coefficient: 0.987; 95% C.I.: 0.942, 0.997). The performance level of DL-MO was comparable to that of the grouped radiologists, i.e. the mean performance difference was -3.3%. The CHO performance was poorer than the grouped radiologist performance, before internal noise could be added. The correlation between CHO and radiologists was weaker (Pearson's correlation coefficient: 0.812, and 95% C.I.: [0.378, 0.955]), and the corresponding performance bias (-29.5%) was statistically significant. CONCLUSION The presented study demonstrated the potential of using the DL-MO for image quality assessment in patient abdominal CT tasks. This article is protected by copyright. All rights reserved.

[1]  Qiyuan Hu,et al.  Deep-learning-based model observer for a lung nodule detection task in computed tomography , 2020, Journal of medical imaging.

[2]  Young-Wook Choi,et al.  Deep learning model observer for 4-alternative forced choice in digital breast tomosynthesis , 2020 .

[3]  Hilde Bosmans,et al.  Deep learning channelized Hotelling observer for multi-vendor DBT system image quality evaluation , 2020, Medical Imaging.

[4]  Craig K. Abbey,et al.  Foveated model observer to predict human search performance on virtual digital breast tomosynthesis phantoms , 2020, Medical Imaging.

[5]  Byeongjoon Kim,et al.  A Convolutional Neural Network-Based Anthropomorphic Model Observer for Signal Detection in Breast CT Images Without Human-Labeled Data , 2020, IEEE Access.

[6]  Baiyu Chen,et al.  Localization of liver lesions in abdominal CT imaging: II. Mathematical model observer performance correlates with human observer performance for localization of liver lesions in abdominal CT imaging , 2019, Physics in medicine and biology.

[7]  Baiyu Chen,et al.  Localization of liver lesions in abdominal CT imaging: I. Correlation of human observer performance between anatomical and uniform backgrounds , 2019, Physics in medicine and biology.

[8]  Hua Li,et al.  Approximating the Ideal Observer and Hotelling Observer for Binary Signal Detection Tasks by Use of Supervised Learning Methods , 2019, IEEE Transactions on Medical Imaging.

[9]  Anaïs Viry,et al.  Channelized Hotelling observer correlation with human observers for low-contrast detection in liver CT images , 2019, Journal of medical imaging.

[10]  Wei Zhou,et al.  A deep learning- and partial least square regression-based model observer for a low-contrast lesion detection task in CT. , 2019, Medical physics.

[11]  Premkumar Elangovan,et al.  Using transfer learning for a deep learning model observer , 2019, Medical Imaging.

[12]  K. Togashi,et al.  Imaging Characteristics of Liver Metastases Overlooked at Contrast-Enhanced CT. , 2019, AJR. American journal of roentgenology.

[13]  Lifeng Yu,et al.  Correlation between a deep-learning-based model observer and human observer for a realistic lung nodule localization task in chest CT , 2019, Medical Imaging.

[14]  Felix K. Kopp,et al.  CNN as model observer in a liver lesion detection task for x‐ray computed tomography: A phantom study , 2018, Medical physics.

[15]  Premkumar Elangovan,et al.  A deep learning model observer for use in alterative forced choice virtual clinical trials , 2018, Medical Imaging.

[16]  Matthew A Kupinski,et al.  Correlation between a 2D channelized Hotelling observer and human observers in a low‐contrast detection task with multislice reading in CT , 2017, Medical physics.

[17]  Shuai Leng,et al.  Estimation of Observer Performance for Reduced Radiation Dose Levels in CT: Eliminating Reduced Dose Levels That Are Too Low Is the First Step. , 2017, Academic radiology.

[18]  Francesc Massanes,et al.  Evaluation of CNN as anthropomorphic model observer , 2017, Medical Imaging.

[19]  Miguel P Eckstein,et al.  Foveated model observers to predict human performance in 3D images , 2017, Medical Imaging.

[20]  Qiyuan Hu,et al.  A virtual clinical trial using projection-based nodule insertion to determine radiologist reader performance in lung cancer screening CT , 2017, Medical Imaging.

[21]  R. Morin,et al.  U.S. Diagnostic Reference Levels and Achievable Doses for 10 Adult CT Examinations. , 2017, Radiology.

[22]  Chi Wan Koo,et al.  Evaluation of a projection-domain lung nodule insertion technique in thoracic CT , 2016, SPIE Medical Imaging.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Shuai Leng,et al.  Lesion insertion in the projection domain: Methods and initial results. , 2015, Medical physics.

[25]  F O Bochud,et al.  Image quality in CT: From physical measurements to model observers. , 2015, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[26]  W J H Veldkamp,et al.  Low contrast detectability performance of model observers based on CT phantom images: kVp influence. , 2015, Physica medica : PM : an international journal devoted to the applications of physics to medicine and biology : official journal of the Italian Association of Biomedical Physics.

[27]  Michael A. Bruno,et al.  Understanding and Confronting Our Mistakes: The Epidemiology of Error in Radiology and Strategies for Error Reduction. , 2015, Radiographics : a review publication of the Radiological Society of North America, Inc.

[28]  Shuai Leng,et al.  Assessment of Low-Contrast Resolution for the American College of Radiology Computed Tomographic Accreditation Program: What Is the Impact of Iterative Reconstruction? , 2015, Journal of computer assisted tomography.

[29]  Mitsuru Ikeda,et al.  Lung nodule detection performance in five observers on computed tomography (CT) with adaptive iterative dose reduction using three-dimensional processing (AIDR 3D) in a Japanese multicenter study: Comparison between ultra-low-dose CT and low-dose CT by receiver-operating characteristic analysis. , 2015, European journal of radiology.

[30]  Yi Zhang,et al.  Degradation of CT Low-Contrast Spatial Resolution Due to the Use of Iterative Reconstruction and Reduced Dose Levels. , 2015, Radiology.

[31]  Shuai Leng,et al.  Lesion insertion in projection domain for computed tomography image quality assessment , 2015, Medical Imaging.

[32]  Ehsan Samei,et al.  An Improved Index of Image Quality for Task-based Performance of CT Iterative Reconstruction across Three Commercial Implementations. , 2015, Radiology.

[33]  Ehsan Samei,et al.  Quantum noise properties of CT images with anatomical textured backgrounds across reconstruction algorithms: FBP and SAFIRE. , 2014, Medical physics.

[34]  H C Gifford Efficient visual-search model observers for PET. , 2014, The British journal of radiology.

[35]  W J H Veldkamp,et al.  Comparison between human and model observer performance in low-contrast detection tasks in CT images: application to images reconstructed with filtered back projection and iterative algorithms. , 2014, The British journal of radiology.

[36]  J. Shindoh,et al.  Three-dimensional volumetry in 107 normal livers reveals clinically relevant inter-segment variation in size. , 2014, HPB : the official journal of the International Hepato Pancreato Biliary Association.

[37]  R. Brereton,et al.  Partial least squares discriminant analysis: taking the magic away , 2014 .

[38]  Ke Li,et al.  Statistical model based iterative reconstruction (MBIR) in clinical CT systems: experimental assessment of noise performance. , 2014, Medical physics.

[39]  Howard C Gifford A visual-search model observer for multislice-multiview SPECT images. , 2013, Medical physics.

[40]  Trafton Drew,et al.  Scanners and drillers: characterizing expert visual search through volumetric images. , 2013, Journal of vision.

[41]  Shuai Leng,et al.  Prediction of human observer performance in a 2-alternative forced choice low-contrast detection task using channelized Hotelling observer: impact of radiation dose and reconstruction algorithms. , 2013, Medical physics.

[42]  M. Shiung,et al.  Development and Validation of a Practical Lower-Dose-Simulation Tool for Optimizing Computed Tomography Scan Protocols , 2012, Journal of computer assisted tomography.

[43]  Adam Wunderlich,et al.  A Nonparametric Procedure for Comparing the Areas Under Correlated LROC Curves , 2012, IEEE Transactions on Medical Imaging.

[44]  Shuai Leng,et al.  Correlation between model observer and human observer performance in CT imaging when lesion location is uncertain , 2012, Medical Imaging.

[45]  Grace J Gang,et al.  Analysis of Fourier-domain task-based detectability index in tomosynthesis and cone-beam CT in relation to human observer performance. , 2011, Medical physics.

[46]  L. Fayad,et al.  Soft-tissue masses and masslike conditions: what does CT add to diagnosis and management? , 2010, AJR. American journal of roentgenology.

[47]  G. Moneta Radiation Dose Associated With Common Computed Tomography Examinations and the Associated Lifetime Attributable Risk of Cancer , 2010 .

[48]  D. Miglioretti,et al.  Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. , 2009, Archives of internal medicine.

[49]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Jia Deng,et al.  A large-scale hierarchical image database , 2009, CVPR 2009.

[51]  Lucretiu M Popescu,et al.  Nonparametric ROC and LROC analysis. , 2007, Medical physics.

[52]  Richard Simon,et al.  Bias in error estimation when using cross-validation for model selection , 2006, BMC Bioinformatics.

[53]  Wenjiang J. Fu,et al.  Estimating misclassification error with small samples via bootstrap cross-validation , 2005, Bioinform..

[54]  E. Samei,et al.  Effects of Anatomical Structure on Signal Detection , 2000 .

[55]  E Samei,et al.  Detection of subtle lung nodules: relative influence of quantum and anatomic noise on chest radiographs. , 1999, Radiology.

[56]  Francis R. Verdun,et al.  Importance of anatomical noise in mammography , 1997, Medical Imaging.

[57]  R. Swensson Unified measurement of observer performance in detecting and localizing target objects on images. , 1996, Medical physics.

[58]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[59]  R D Nawfel,et al.  Contrast-detail curves for liver CT. , 1992, Medical physics.

[60]  H L Kundel,et al.  Nodule detection with and without a chest image. , 1985, Investigative radiology.