Questions Program Directors Need to Answer Before Using Resident Clinical Performance Data.

T wice a year, training programs must report milestones for every resident to the Accreditation Council for Graduate Medical Education (ACGME). The ACGME lists possible resident progress assessment methods to inform the milestones, but many are subjective. In addition, the ACGME surveys residents to verify that programs give trainees feedback on their performance, as well as their personal clinical effectiveness. In an effort to make feedback in the latter dimension reliable and meaningful, program directors are searching for and devising systems to give objective, unbiased clinical performance data. The ability to gather and report process and outcome data via automated systems (eg, electronic health records, registries, and billing data) in medical practice is relatively new, and educators should be aware of the complexities. Obtaining structured, objective clinical performance feedback data can be a challenge. Some groups provide automatic feedback of clinical performance data on measures like proper antibiotic administration and incidence of complications. Unfortunately, the authors of 1 study were unable to find a correlation between the level of training and the performance on these metrics, or any longitudinal improvement in the metrics for a given resident over time. As departments collect data for quality and milestone reporting, this should allow them to parse the data to the level of individual residents. The temptation to use these data to ‘‘get some numbers,’’ to meaningfully fulfill the feedback requirement, may become significant. This secondary use of patient data from electronic health records, billing, and other sources to understand individual provider performance is still in its infancy, and data can easily be misinterpreted and misused. Accuracy and transparency must be considered before providing residents with data gathered for other purposes, and particularly before using it for competency determinations. When devising policies to use data gathered for other purposes to evaluate resident clinical performance, program directors should be prepared to answer the following questions. 1. How can we be sure that these data reflect a specific resident’s patients?