High fidelity simulation can discriminate between novice and experienced residents when assessing competency in patient care

Background: High fidelity (HF) simulators have become more common in residency training programs. Aims: We hypothesized that high fidelity (HF) simulation-based assessment of patient care competency could differentiate novice from experienced residents. Methods: Prospective study of 44 emergency medicine residents. A simulated case of anaphylactic shock was administered to each participating resident utilizing a HF patient simulator. Management of the case required epinephrine, airway management including a surgical airway, and IV fluids. Data was recorded using a standardized form and stop watch. Primary outcome: Time to completion of surgical airway. Secondary outcomes: Times to administration of epinephrine/attempt intubation/start surgical airway/complete case; checklist items: epinephrine as first action, pre-oxygenation, cricoid pressure and type of surgical airway. Results: Novice residents took significantly longer than the experienced residents to achieve our primary outcome, time to completion of surgical airway (621/512 sec; p = 0.03). The novice residents took significantly longer to acheive three of our secondary outcomes: time to start of surgical airway (534 versus 442 sec; p = 0.04), time to case completion (650 versus 513 sec; p = 0.006), and epinephrine as a first action (73% versus 100%; p = 0.02). Conclusions: HF simulation-based assessment using objective measures, particularly time to action, discerned our novice from our experienced residents. Practice points An assessment tool that can differentiate between different levels of competency demonstrates construct validity. Reports of the validity and reliability of simulator-based assessments of resident performance are limited and call for further research in this sector of medical education. The Patient Care competency encompasses data gathering, integration of diagnostic information, implementation of an effective patient management plan, competent performance of procedures and the ability to prioritize and stabilize multiple patients. Because of the complexity, Patient Care competency may be more difficult to measure than other competencies such as Medical Knowledge. HF simulation may be a useful tool for evaluating this competency. Our study demonstrated that a HF simulation-based asssessment using objective measures, particularly time to action, discerned our novice from our experienced residents. This finding of construct validity adds to the growing body of evidence that HF simulation may become a valid method of assessing Patient Care competency in residents.