A VISION Substudy of Reader Agreement on 68Ga-PSMA-11 PET/CT Scan Interpretation to Determine Patient Eligibility for 177Lu-PSMA-617 Radioligand Therapy

Visual Abstract [ 68Ga]Ga-PSMA-11 ( 68Ga-PSMA-11) is used to identify prostate-specific membrane antigen (PSMA)–positive tumors on PET scans. In the VISION study, 68Ga-PSMA-11 was used to determine the eligibility of patients with metastatic castration-resistant prostate cancer for treatment with [177Lu]Lu-PSMA-617 (177Lu-PSMA-617), based on predefined read criteria. This substudy aimed to investigate the interreader variability and intrareader reproducibility of visual assessments of 68Ga-PSMA-11 PET/CT scans using the VISION read criteria and evaluate the agreement between read results for this and the VISION study. Methods: In VISION, 68Ga-PSMA-11 PET/CT scans were centrally read as inclusion cases if they had at least 1 PSMA-positive lesion and no PSMA-negative lesions that fulfilled the exclusion criteria. In this substudy, 125 PET/CT scans (75 inclusion and 50 exclusion cases) were randomly selected from VISION and retrospectively assessed by 3 independent central readers. A random subset of 20 cases (12 inclusion and 8 exclusion cases) was recoded for assessment of intrareader reproducibility. Classification of cases as inclusion or exclusion cases was based on the VISION read criteria. Overall interreader variability was assessed by Fleiss κ-statistics, and pairwise variability and intrareader reproducibility were assessed by Cohen κ-statistics. Results: For interreader variability, the readers agreed on 77% of cases (overall average agreement rate, 0.85; Fleiss κ, 0.60 [95% CI, 0.50–0.70]). The pairwise agreement rate was 0.82, 0.88, and 0.84, and the corresponding Cohen κ was 0.54 (95% CI, 0.38–0.71), 0.67 (95% CI, 0.52–0.83), and 0.59 (95% CI, 0.43–0.75), respectively. For intrareader reproducibility, the agreement rate was 0.90, 0.90, and 0.95, and the corresponding Cohen κ was 0.78 (95% CI, 0.49–0.99), 0.76 (95% CI, 0.46–0.99), and 0.89 (95% CI, 0.67–0.99), respectively. The number of actual VISION inclusion cases out of the total number of cases scored as inclusion in this substudy was 71 of 93 (agreement rate, 0.76; 95% CI, 0.66–0.85) for reader 1, 70 of 88 (0.80; 0.70–0.87) for reader 2, and 73 of 96 (0.76; 0.66–0.84) for reader 3. All readers agreed on 66 of 75 VISION inclusion cases. Conclusion: Moderate-to-substantial interreader agreement and substantial-to-almost perfect intrareader reproducibility for 68Ga-PSMA-11 PET/CT scan assessment using the VISION read criteria were observed. The read rules applied in VISION can be readily learned and demonstrate good reproducibility.