Prospective assessment of an atlas-based intervention combined with real-time software feedback in contouring lymph node levels and organs-at-risk in the head and neck: Quantitative assessment of conformance to expert delineation.

PURPOSE A number of studies have previously assessed the role of teaching interventions to improve organ-at-risk (OAR) delineation. We present a preliminary study demonstrating the benefit of a combined atlas and real time software-based feedback intervention to aid in contouring of OARs in the head and neck. METHODS AND MATERIALS The study consisted of a baseline evaluation, a real-time feedback intervention, atlas presentation, and a follow-up evaluation. At baseline evaluation, 8 resident observers contoured 26 OARs on a computed tomography scan without intervention or aid. They then received feedback comparing their contours both statistically and graphically to a set of atlas-based expert contours. Additionally, they received access to an atlas to contour these structures. The resident observers were then asked to contour the same 26 OARs on a separate computed tomography scan with atlas access. In addition, 6 experts (5 radiation oncologists specializing in the head and neck, and 1 neuroradiologist) contoured the 26 OARs on both scans. A simultaneous truth and performance level estimation (STAPLE) composite of the expert contours was used as a gold-standard set for analysis of OAR contouring. RESULTS Of the 8 resident observers who initially participated in the study, 7 completed both phases of the study. Dice similarity coefficients were calculated for each user-drawn structure relative to the expert STAPLE composite for each structure. Mean dice similarity coefficients across all structures increased between phase 1 and phase 2 for each resident observer, demonstrating a statistically significant improvement in overall OAR-contouring ability (P < .01). Additionally, intervention improved contouring in 16/26 delineated organs-at-risk across resident observers at a statistically significant level (P ≤ .05) including all otic structures and suprahyoid lymph node levels of the head and neck. CONCLUSIONS Our data suggest that a combined atlas and real-time feedback-based educational intervention detectably improves contouring of OARs in the head and neck.

[1]  Niko Papanikolaou,et al.  Prospective randomized double-blind pilot study of site-specific consensus atlas implementation for rectal cancer target volume delineation in the cooperative group setting. , 2011, International journal of radiation oncology, biology, physics.

[2]  Johannes A Langendijk,et al.  Delineation guidelines for organs at risk involved in radiation-induced salivary dysfunction and xerostomia. , 2009, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[3]  William M. Wells,et al.  Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation , 2004, IEEE Transactions on Medical Imaging.

[4]  Wolfgang A Tomé,et al.  Heterogeneity in head and neck IMRT target design and clinical practice. , 2012, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[5]  Marcel van Herk,et al.  Target definition in prostate, head, and neck. , 2005, Seminars in radiation oncology.

[6]  M. Stock,et al.  Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy , 2012, Strahlentherapie und Onkologie.

[7]  Clifton D Fuller,et al.  Report from the Radiation Therapy Committee of the Southwest Oncology Group (SWOG): Research Objectives Workshop 2008 , 2009, Clinical Cancer Research.

[8]  Clifton D Fuller,et al.  A pilot prospective feasibility study of organ-at-risk definition using Target Contour Testing/Instructional Computer Software (TaCTICS), a training and evaluation platform for radiotherapy target delineation. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[9]  K. Ang,et al.  CT-based delineation of lymph node levels and related CTVs in the node-negative neck: DAHANCA, EORTC, GORTEC, NCIC,RTOG consensus guidelines. , 2003, Radiotherapy and oncology : journal of the European Society for Therapeutic Radiology and Oncology.

[10]  W. Woodward,et al.  Variability of target and normal structure delineation for breast cancer radiotherapy: an RTOG Multi-Institutional and Multiobserver Study. , 2007, International journal of radiation oncology, biology, physics.

[11]  Edward A. Feustel,et al.  The asymptotic relative efficiency of mixed statistical tests , 1967, IEEE Trans. Inf. Theory.

[12]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[13]  Hilke Vorwerk,et al.  Guidelines for delineation of lymphatic clinical target volumes for high conformal radiotherapy: head and neck region , 2011, Radiation oncology.

[14]  J. Roeske,et al.  Survey of Resident Education in Intensity-Modulated Radiation Therapy , 2005, Technology in cancer research & treatment.

[15]  W. Mendenhall,et al.  Contouring the Middle and Inner Ear on Radiotherapy Planning Scans , 2005, American journal of clinical oncology.

[16]  Nancy Lee,et al.  Head-and-neck target delineation among radiation oncology residents after a teaching intervention: a prospective, blinded pilot study. , 2009, International journal of radiation oncology, biology, physics.

[17]  Clifton David Fuller,et al.  Target Contour Testing/Instructional Computer Software (TaCTICS): A Novel Training and Evaluation Platform for Radiotherapy Target Delineation. , 2010, AMIA ... Annual Symposium proceedings. AMIA Symposium.