Evaluation of a method of computer-aided detection (CAD) of pulmonary nodules at computed tomography

PurposeThe authors sought to compare the sensitivity and reading time obtained using computer-aided detection (CAD) software as second reader (SR) or concurrent reader (CR) in the identification of pulmonary nodules.Materials and methodsUnenhanced CT scans of 100 consecutive cancer patients were retrospectively reviewed by four readers to identify all solid, noncalcified pulmonary nodules ranging from 3 to 30 mm in diameter. The sensitivity and reading time of each reader and of CAD alone were calculated at 3-mm and 5-mm thresholds with respect to the reference standard, consisting of a consensus reading by the four radiologists involved in the study. The McNemar test was used to compare the sensitivities obtained by reading without CAD (readers 1 and 2), with CAD as SR (readers 1 and 2 with a 2-month delay), and with CAD as CR (readers 3 and 4). The paired Student’s t test was used to compare reading times. A value of p<0.05 was considered statistically significant.ResultsA total of 258 and 224 nodules were identified at 3-mm and 5-mm thresholds, respectively. The sensitivity of CAD alone was 62.79% and 67.41% at the 3-mm and 5-mm threshold values respectively, with 4.15 and 2.96 false-positive findings per examination. CAD as SR produced a significant increase in sensitivity (p<0.001) in nodule detection with respect to reading without CAD both at 3 mm (12.01%) and 5 mm (10.04%); the average increase in sensitivity obtained when comparing CAD as SR to CAD as CR was statistically significant (p<0.025) both at the 3-mm (5.35%) and 5-mm (4.68%) thresholds. CAD as CR produced a nonsignificant increase in sensitivity compared with reading without CAD (p>0.05). Mean reading time using CAD as SR (330 s) was significantly longer than reading without CAD (135 s, p<0.001) and reading with CAD as CR (195 s, p<0.025).ConclusionsThe use of CAD as CR, without any significant increase in reading time, produces no significant increase in sensitivity in pulmonary nodule detection when compared with reading without CAD (p>0.05); CAD as SR, at the cost of longer reading times, increases sensitivity when compared with reading without CAD (p<0.001) or with CAD as CR (p<0.025).RiassuntoObiettivoScopo del nostro lavoro è stato confrontare la sensibilità ed i tempi di lettura ottenuti mediante il sistema diagnosi computer-assistita (CAD) della nostra apparecchiatura per la tomografia computerizzata multistrato (TCMS) in qualità di secondo lettore (SL) e di lettore concorrente (LC) nella identificazione dei noduli polmonari.Materiale e metodiLe TCMS non contrastografiche di 100 pazienti oncologici consecutivi sono state retrospettivamente analizzate da 4 lettori alla ricerca di noduli non calcifici di diametro compreso tra 3 e 30 mm. Utilizzando come standard di riferimento la lettura in consenso effettuata da 4 radiologi, sono stati calcolati sensibilità e tempi di lettura di ogni lettore e del CAD da solo, ai valori soglia dei noduli di 3 e 5 mm. È stato utilizzato il test di McNemar per confrontare i valori di sensibilità ottenuti dalla lettura senza CAD (lettori 1 e 2), con CAD come SL (lettori 1 e 2 dopo 2 mesi), con CAD come LC (lettori 3 e 4). Il test t di Student è stato utilizzato per confrontare i tempi di lettura. I valori statistici sono stati considerati significativi per p<0,05.RisultatiSono stati identificati 258 e 224 noduli rispettivamente alla soglia di 3 e 5 mm. Il CAD da solo ha presentato una sensibilità del 62,79% e 67,41% ai valori soglia di 3 e 5 mm, rispettivamente con 4,15 e 2,96 falsi positivi per esame. L’utilizzo del CAD come SL ha favorito un significativo aumento della sensibilità (p<0,001) nell’identificazione dei noduli polmonari rispetto alla lettura senza CAD sia alla soglia di 3 mm (12,01%) che a quella di 5 mm (10,04%); l’aumento medio di sensibilità ottenuto dal confronto della lettura con CAD come SL e CAD come LC è stata statisticamente significativa (p<0,025) sia alla soglia di 3 mm (5,35%) che di 5 mm (4,68%). Il CAD come LC ha favorito un aumento statisticamente non significativo della sensibilità se confrontato alla lettura senza CAD (p>0,05). I tempi di lettura utilizzando il CAD come SL (330 secondi) sono stati significativamente più lunghi rispetto alla lettura senza CAD (135 secondi, p<0,001) e alla lettura con CAD come LC (195 secondi, p<0,025).ConclusioniL’utilizzo del CAD come LC, senza un significativo aumento dei tempi di lettura, non comporta un aumento significativo della sensibilità nell’identificazione dei noduli polmonari rispetto alla lettura senza CAD (p>0,05); l’utilizzo del CAD come SL, a prezzo di un allungamento dei tempi di lettura, garantisce un significativo aumento della sensibilità rispetto alla lettura senza CAD (p<0,001) e con CAD come LC (p<0,025).

[1]  Iva Petkovska,et al.  Computer-aided Diagnosis in Lung Nodule Assessment , 2008, Journal of thoracic imaging.

[2]  R W Günther,et al.  Performance evaluation of a computer-aided detection algorithm for solid pulmonary nodules in low-dose and standard-dose MDCT chest examinations and its influence on radiologists. , 2008, The British journal of radiology.

[3]  Margrit Betke,et al.  Chest CT: automated nodule detection and assessment of change over time--preliminary experience. , 2001, Radiology.

[4]  W. Heindel,et al.  Volumetric measurements of pulmonary nodules at multi-row detector CT: in vivo reproducibility , 2003, European Radiology.

[5]  K. Awai,et al.  Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. , 2004, Radiology.

[6]  Peter Herzog,et al.  Computer-aided diagnosis as a second reader: spectrum of findings in CT studies of the chest interpreted as normal. , 2005, Chest.

[7]  Annette McWilliams,et al.  Computed tomography-detected noncalcified pulmonary nodules: a review of evidence for significance and management. , 2008, Proceedings of the American Thoracic Society.

[8]  Binsheng Zhao,et al.  Small pulmonary nodules: volumetrically determined growth rates based on CT evaluation. , 2000, Radiology.

[9]  J E Kuhlman,et al.  Pulmonary nodules: effect of increased data sampling on detection with spiral CT and confidence in diagnosis. , 1995, Radiology.

[10]  S. Armato,et al.  Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. , 2002, Radiology.

[11]  L. Kreel,et al.  Distribution of lung metastases in the axial plane. A combined radiological-pathological study. , 1977, Radiologia clinica.

[12]  P. Grenier,et al.  Computer-aided detection of solid lung nodules on follow-up MDCT screening: evaluation of detection, tracking, and reading time. , 2007, AJR. American journal of roentgenology.

[13]  Ekta Dharaiya,et al.  Lung nodule CAD software as a second reader: a multicenter study. , 2008, Academic radiology.

[14]  Anthony J. Sherbondy,et al.  Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. , 2005, Radiology.

[15]  L. Bonomo,et al.  Automated volumetry of pulmonary nodules on multidetector CT: influence of slice thickness, reconstruction algorithm and tube current. Preliminary results , 2008, La radiologia medica.

[16]  K. Marten,et al.  Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists , 2004, European Radiology.

[17]  H. Tengg-Kobligk,et al.  Image-based evaluation of tumor response to treatment: where is radiology today? , 2008, Experimental oncology.

[18]  Y. Kawata,et al.  Computer-aided diagnosis for pulmonary nodules based on helical CT images , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[19]  Hirofumi Fujii,et al.  Radiologic measurements of tumor response to treatment: practical approaches and limitations. , 2008, Radiographics : a review publication of the Radiological Society of North America, Inc.

[20]  C. Catalano,et al.  Evaluation of effectiveness of a computer system (CAD) in the identification of lung nodules with low-dose MSCT: scanning technique and preliminary results. , 2005, La Radiologia medica.

[21]  K. Marten,et al.  Automated CT volumetry of pulmonary metastases: the effect of a reduced growth threshold and target lesion number on the reliability of therapy response assessment using RECIST criteria , 2007, European Radiology.

[22]  W. Heindel,et al.  Detection of pulmonary nodules at multirow-detector CT: effectiveness of double reading to improve sensitivity at standard-dose and low-dose chest CT , 2004, European Radiology.

[23]  F. Fischbach,et al.  Detection of pulmonary nodules by multislice computed tomography: improved detection rate with reduced slice thickness , 2003, European Radiology.

[24]  H Rusinek,et al.  Variables Affecting Pulmonary Nodule Detection with Computed Tomography: Evaluation with Three‐Dimensional Computer Simulation , 1993, Journal of thoracic imaging.

[25]  L. Broemeling,et al.  Interobserver and intraobserver variability in measurement of non-small-cell carcinoma lung lesions: implications for assessment of tumor response. , 2003, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[26]  G. Angelelli,et al.  Multi slice computed tomography in the study of pulmonary metastases , 2008, La radiologia medica.

[27]  S. Armato,et al.  Lung cancers missed at low-dose helical CT screening in a general population: comparison of clinical, histopathologic, and imaging findings. , 2002, Radiology.

[28]  P. Cooperberg,et al.  Computer-aided detection in screening CT for pulmonary nodules. , 2006, AJR. American journal of roentgenology.

[29]  K. Marten,et al.  Computer-aided detection and automated CT volumetry of pulmonary nodules , 2007, European Radiology.

[30]  K. Doi,et al.  Current status and future potential of computer-aided diagnosis in medical imaging. , 2005, The British journal of radiology.

[31]  S. Swensen,et al.  Screening for lung cancer with low-dose spiral computed tomography. , 2000, American journal of respiratory and critical care medicine.

[32]  S. Armato,et al.  Computerized detection of pulmonary nodules on CT scans. , 1999, Radiographics : a review publication of the Radiological Society of North America, Inc.

[33]  L. Schwartz,et al.  Evaluation of tumor measurements in oncology: use of film-based and electronic techniques. , 2000, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[34]  Geoffrey McLennan,et al.  Assessment of radiologist performance in the detection of lung nodules: dependence on the definition of "truth". , 2009, Academic radiology.

[35]  M. Giger,et al.  Computerized Detection of Pulmonary Nodules in Computed Tomography Images , 1994, Investigative radiology.

[36]  W. Heindel,et al.  Comparison of sensitivity and reading time for the use of computer-aided detection (CAD) of pulmonary nodules at MDCT as concurrent or second reader , 2007, European Radiology.