A Method for Designing Symmetrical Simulation Scenarios for Evaluation of Behavioral Skills

The increasing use of simulation technology in medical education necessitates a method to formulate carefully planned scenarios to assess performance. Much of simulation in healthcare endeavors to teach technical skills such as the drugs and dosages needed during Advanced Cardiac Life Support (ACLS)1 or procedural skills such as those needed for laparoscopic surgery2– 8 or bronchoscopy.9 –11 In contrast, scenarios can be designed specifically to evaluate behavioral skills such as communication, leadership, and situation awareness. These are skills typically emphasized in crisis resource management (CRM) courses.12–15 This manuscript describes a systematic framework for developing interchangeable, “symmetrical” simulation scenarios that can be randomly ordered to repeatedly test crisis behavioral skills. The mechanism given in this methodologic description is reproducible and can generate a series of scenarios that are similar in flow, basic content, and difficulty. This standardizes the expertise necessary to solve each crisis and avoids the wide range of complexity commonly seen in simulation scenarios. Symmetrical scenarios provide a context for assessment using behavioral skills scoring systems by providing clear opportunities within each scenario for participants to demonstrate elements of each skill. To compare “before and after” performance and behavioral skills retention using such a rating system, it is essential to have comparable simulation scenarios. Many institutions derive their CRM scenarios from a database of clinical scenarios already in existence. Scenarios developed from actual cases may be the most realistic and can avoid a potential complaint that “this would never happen in real life.” Some experts would argue that all scenarios should be based on actual clinical events for this reason, and it is reasonable to assume that purely formative sessions might best be grounded in specific clinical experiences. However, restricting simulations to known clinical events may lead to widely disparate scenarios, making comparisons of technical and behavioral skills difficult. A recent study by Yee and coauthors assessed and reevaluated crisis behavioral skills in anesthesiology residents using clinical scenarios from an existing repertoire.16 Using a variety of clinical scenarios with a diverse range of complexity, they found a measurable benefit from CRM training but could not show additional benefit from further simulator sessions. When assessing behavioral skills with complex, catastrophic crises, it may be difficult to distinguish poor performance from flawed scenarios.17 Indeed, an early investigation by Gaba et al. in 1998 stated a need for “more tightly controlled scenarios.”17 This proposed method of symmetrical scenario development relies on actual clinical experiences adapted in a realistic but specific manner to elicit particular responses from team From the Simulation Development and Cognitive Science Laboratory, Office of Education Affairs, Pennsylvania State University College of Medicine, Hershey, PA.