An Assessment of the Quality Oncology Practice Initiative: Lessons Learned From a Detailed Assessment of a Well-Established Profession-Based Performance Measurement Program

Background: National performance measurement programs used to improve quality and/or accountability are prevalent. Professional society-based programs largely assess practice or provider performance. We tested the limitations of the Quality Oncology Practice Initiative (QOPI), a well-established program. Methods: We investigated the potential limitations of the QOPI limited sampling strategy and end-of-life (EOL) metrics through a retrospective review of all decedents meeting the QOPI eligibility criteria at a large three-site community oncology practice. Relative precision for each EOL metric was measured and simulated scenarios modeled via log-normal distributions were compared. Results: A total of 246 deaths were identified; only 14% of decedents were included in the QOPI limited sample. Important differences in relative precision between samples were evident. Chemotherapy administered at the EOL was 2.8 times greater among the full cohort compared to the sample. Conclusion: The limited sampling strategy demonstrated the lack of precision explained by sampling variability confounded in smaller sample sizes. Our analyses demonstrated that practices reporting the same metric may stem from very different underlying distributions, which limits the ability to understand complex practice patterns and improve care. These findings may be of significance to other performance measurement programs that utilize similar testing strategies.