Low‐dose CT for the detection and classification of metastatic liver lesions: Results of the 2016 Low Dose CT Grand Challenge

Purpose: Using common datasets, to estimate and compare the diagnostic performance of image‐based denoising techniques or iterative reconstruction algorithms for the task of detecting hepatic metastases. Methods: Datasets from contrast‐enhanced CT scans of the liver were provided to participants in an NIH‐, AAPM‐ and Mayo Clinic‐sponsored Low Dose CT Grand Challenge. Training data included full‐dose and quarter‐dose scans of the ACR CT accreditation phantom and 10 patient examinations; both images and projections were provided in the training data. Projection data were supplied in a vendor‐neutral standardized format (DICOM‐CT‐PD). Twenty quarter‐dose patient datasets were provided to each participant for testing the performance of their technique. Images were provided to sites intending to perform denoising in the image domain. Fully preprocessed projection data and statistical noise maps were provided to sites intending to perform iterative reconstruction. Upon return of the denoised or iteratively reconstructed quarter‐dose images, randomized, blinded evaluation of the cases was performed using a Latin Square study design by 11 senior radiology residents or fellows, who marked the locations of identified hepatic metastases. Markings were scored against reference locations of clinically or pathologically demonstrated metastases to determine a per‐lesion normalized score and a per‐case normalized score (a faculty abdominal radiologist established the reference location using clinical and pathological information). Scores increased for correct detections; scores decreased for missed or incorrect detections. The winner for the competition was the entry that produced the highest total score (mean of the per‐lesion and per‐case normalized score). Reader confidence was used to compute a Jackknife alternative free‐response receiver operating characteristic (JAFROC) figure of merit, which was used for breaking ties. Results: 103 participants from 90 sites and 26 countries registered to participate. Training data were shared with 77 sites that completed the data sharing agreements. Subsequently, 41 sites downloaded the 20 test cases, which included only the 25% dose data (CTDIvol = 3.0 ± 1.8 mGy, SSDE = 3.5 ± 1.3 mGy). 22 sites submitted results for evaluation. One site provided binary images and one site provided images with severe artifacts; cases from these sites were excluded from review and the participants removed from the challenge. The mean (range) per‐lesion and per‐case normalized scores were −24.2% (−75.8%, 3%) and 47% (10%, 70%), respectively. Compared to reader results for commercially reconstructed quarter‐dose images with no noise reduction, 11 of the 20 sites showed a numeric improvement in the mean JAFROC figure of merit. Notably two sites performed comparably to the reader results for full‐dose commercial images. The study was not designed for these comparisons, so wide confidence intervals surrounded these figures of merit and the results should be used only to motivate future testing. Conclusion: Infrastructure and methodology were developed to rapidly estimate observer performance for liver metastasis detection in low‐dose CT examinations of the liver after either image‐based denoising or iterative reconstruction. The results demonstrated large differences in detection and classification performance between noise reduction methods, although the majority of methods provided some improvement in performance relative to the commercial quarter‐dose images with no noise reduction applied.

[1]  A. Beckett,et al.  AKUFO AND IBARAPA. , 1965, Lancet.

[2]  J M Bland,et al.  Statistical methods for assessing agreement between two methods of clinical measurement , 1986 .

[3]  K. Berbaum,et al.  Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. , 1992, Investigative radiology.

[4]  Dev P Chakraborty,et al.  Observer studies involving detection and localization: modeling, analysis, and validation. , 2004, Medical physics.

[5]  Cynthia H McCollough,et al.  The phantom portion of the American College of Radiology (ACR) computed tomography (CT) accreditation program: practical tips, artifact examples, and pitfalls to avoid. , 2004, Medical physics.

[6]  C. McCollough,et al.  CT dose reduction and dose management tools: overview of available options. , 2006, Radiographics : a review publication of the Radiological Society of North America, Inc.

[7]  S. Hillis A comparison of denominator degrees of freedom methods for multiple observer ROC analysis , 2007, Statistics in medicine.

[8]  C. McCollough,et al.  Radiation dose reduction in computed tomography: techniques and future perspective. , 2009, Imaging in medicine.

[9]  T. Yoshizumi,et al.  Radiologic and nuclear medicine studies in the United States and worldwide: frequency, radiation dose, and comparison with other radiation sources--1950-2007. , 2009, Radiology.

[10]  Cynthia H McCollough,et al.  Managing radiation use in medical imaging: a multifaceted challenge. , 2011, Radiology.

[11]  A. Einstein,et al.  Impact of reduced patient life expectancy on potential cancer risks from radiologic imaging. , 2011, Radiology.

[12]  John M Boone,et al.  Radiation exposure from CT scans: how to close our knowledge gaps, monitor and safeguard exposure--proceedings and recommendations of the Radiation Dose Summit, sponsored by NIBIB, February 24-25, 2011. , 2012, Radiology.

[13]  Katsuyuki Taguchi,et al.  Achieving routine submillisievert CT scanning: report from the summit on management of radiation dose in CT. , 2012, Radiology.

[14]  M. Shiung,et al.  Development and Validation of a Practical Lower-Dose-Simulation Tool for Optimizing Computed Tomography Scan Protocols , 2012, Journal of computer assisted tomography.

[15]  Patrik Rogalla,et al.  Iterative reconstruction algorithm for CT: can radiation dose be decreased while low-contrast detectability is preserved? , 2013, Radiology.

[16]  E. Samei,et al.  Clinical impact of an adaptive statistical iterative reconstruction algorithm for detection of hypervascular liver tumours using a low tube voltage, high tube current MDCT technique , 2013, European Radiology.

[17]  Nancy A Obuchowski,et al.  Effect of reduced radiation exposure and iterative reconstruction on detection of low-contrast low-attenuation lesions in an anthropomorphic liver phantom: an 18-reader study. , 2014, Radiology.

[18]  Rochester,et al.  Use of Water Equivalent Diameter for Calculating Patient Size and Size-Specific Dose Estimates (SSDE) in CT: The Report of AAPM Task Group 220. , 2014, AAPM report.

[19]  Armando Manduca,et al.  Observer Performance in the Detection and Classification of Malignant Hepatic Nodules and Masses with CT Image-Space Denoising and Iterative Reconstruction. , 2015, Radiology.

[20]  Joel G Fletcher,et al.  Answers to Common Questions About the Use and Safety of CT Scans. , 2015, Mayo Clinic proceedings.

[21]  Xinhui Duan,et al.  Technical Note: Development and validation of an open data format for CT projection data. , 2015, Medical physics.

[22]  Shuai Leng,et al.  An open library of CT patient projection data , 2016, SPIE Medical Imaging.