Script Concordance Testing: Assessing Residents’ Clinical Decision-Making Skills for Infant Lumbar Punctures

Purpose Residents must learn which infants require a lumbar puncture (LP), a clinical decision-making skill (CDMS) difficult to evaluate because of considerable practice variation. The authors created an assessment model of the CDMS to determine when an LP is indicated, taking practice variation into account. The objective was to detect whether script concordance testing (SCT) could measure CDMS competency among residents for performing infant LPs. Method In 2011, using a modified Delphi technique, an expert panel of 14 attending physicians constructed 15 case vignettes (each with 2 to 4 SCT questions) that represented various infant LP scenarios. The authors distributed the vignettes to residents at 10 academic pediatric centers within the International Simulation in Pediatric Innovation, Research, and Education Network. They compared SCT scores among residents of different postgraduate years (PGYs), specialties, training in adult medicine, LP experience, and practice within an endemic Lyme disease area. Results Of 730 eligible residents, 102 completed 47 SCT questions. They could earn a maximum score of 47. Median SCT scores were significantly higher in PGY-3s compared with PGY-1s (difference: 3.0; 95% confidence interval [CI] 1.0–4.9; effect size d = 0.87). Scores also increased with increasing LP experience (difference: 3.3; 95% CI 1.1–5.5) and with adult medicine training (difference: 2.9; 95% CI 0.6–5.0). Residents in Lyme-endemic areas tended to perform more LPs than those in nonendemic areas. Conclusions SCT questions may be useful as an assessment tool to determine CDMS competency among residents for performing infant LPs.

[1]  M. Auerbach,et al.  Qualitative Evaluation of Just-in-Time Simulation-Based Learning: The Learners’ Perspective , 2013, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[2]  A. Roex,et al.  Let’s talk about thinking , 2012, Medical education.

[3]  Lisette van Bruggen,et al.  Preferred question types for computer-based assessment of clinical reasoning: a literature study , 2012, Perspectives on medical education.

[4]  N. Tofil,et al.  Transfer of Simulated Lumbar Puncture Training to the Clinical Setting , 2012, Pediatric emergency care.

[5]  Matthew M Davis,et al.  Variation in lumbar punctures for early onset neonatal sepsis: a nationally representative serial cross-sectional analysis, 2003-2009 , 2012, BMC Pediatrics.

[6]  J. Barsuk,et al.  Simulation-based education with mastery learning improves residents' lumbar puncture skills , 2012, Neurology.

[7]  B. Charlin,et al.  How to construct and implement script concordance tests: insights from a systematic review , 2012, Medical education.

[8]  M. Poon,et al.  A Study of Thrombophilia Testing and Counseling Practices of Family Physicians Using the Script Concordance Method in Calgary, Canada , 2012, Clinical and applied thrombosis/hemostasis : official journal of the International Academy of Clinical and Applied Thrombosis/Hemostasis.

[9]  B. Charlin,et al.  Summative assessment of 5th year medical students’ clinical reasoning by script concordance test: requirements and challenges , 2012, BMC medical education.

[10]  Derhan Chen,et al.  The ethics script concordance test in assessing ethical reasoning , 2012, Medical education.

[11]  T. Nouh,et al.  The script concordance test as a measure of clinical reasoning: a national validation study. , 2012, American journal of surgery.

[12]  I. Philibert,et al.  The next GME accreditation system--rationale and benefits. , 2012, The New England journal of medicine.

[13]  Emil Petrusa,et al.  Tracking Development of Clinical Reasoning Ability Across Five Medical Schools Using a Progress Test , 2011, Academic medicine : journal of the Association of American Medical Colleges.

[14]  B. Charlin,et al.  Online script concordance test for clinical reasoning assessment in otorhinolaryngology: the association between performance and clinical experience. , 2011, Archives of otolaryngology--head & neck surgery.

[15]  David O. Kessler,et al.  A Randomized Trial of Simulation-Based Deliberate Practice for Infant Lumbar Puncture Skills , 2011, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[16]  Edward J. Miech,et al.  Assessing clinical reasoning skills in scenarios of uncertainty: convergent validity for a Script Concordance Test in an emergency medicine clerkship and residency. , 2011, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[17]  Bernard Charlin,et al.  Script concordance testing: a review of published validity evidence , 2011, Medical education.

[18]  J. de Korwin,et al.  Usefulness of the script concordance test in dermatology , 2011, Journal of the European Academy of Dermatology and Venereology : JEADV.

[19]  R. Duvivier,et al.  Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference , 2011, Medical teacher.

[20]  Carla Pruden,et al.  Procedural readiness of pediatric interns: defining novice performance through simulation. , 2010, Journal of graduate medical education.

[21]  E. Fleegler,et al.  Adherence to Guidelines for Managing the Well-Appearing Febrile Infant: Assessment Using a Case-Based, Interactive Survey , 2010, Pediatric emergency care.

[22]  B. Roos,et al.  The Script Concordance Test as a Measure of Clinical Reasoning Skills in Geriatric Urinary Incontinence , 2010, Journal of the American Geriatrics Society.

[23]  J. Jelovsek,et al.  Assessment of intraoperative judgment during gynecologic surgery using the Script Concordance Test. , 2010, American journal of obstetrics and gynecology.

[24]  S. Mintegi,et al.  Well appearing young infants with fever without known source in the Emergency Department: are lumbar punctures always necessary? , 2010, European journal of emergency medicine : official journal of the European Society for Emergency Medicine.

[25]  D. Swanson,et al.  Use of Multimedia on the Step 1 and Step 2 Clinical Knowledge Components of USMLE: A Controlled Trial of the Impact on Item Characteristics , 2009, Academic medicine : journal of the Association of American Medical Colleges.

[26]  Pat Croskerry,et al.  A universal model of diagnostic reasoning. , 2009, Academic medicine : journal of the Association of American Medical Colleges.

[27]  B. Charlin,et al.  Script concordance testing: more cases or more questions? , 2009, Advances in health sciences education : theory and practice.

[28]  R. Goldman,et al.  Practice Variations in the Treatment of Febrile Infants Among Pediatric Emergency Physicians , 2009, Pediatrics.

[29]  S. Downing,et al.  Assessing clinical reasoning in pediatric emergency medicine: validity evidence for a Script Concordance Test. , 2009, Annals of emergency medicine.

[30]  Bradley J Benson,et al.  From the educational bench to the clinical bedside: translating the Dreyfus developmental model to the learning of clinical skills. , 2008, Academic medicine : journal of the Association of American Medical Colleges.

[31]  Bernard Charlin,et al.  Script Concordance Tests: Guidelines for Construction , 2008, BMC Medical Informatics Decis. Mak..

[32]  Objective assessment of surgical decision making in trauma after a laboratory-based course: durability of cognitive skills. , 2008, American journal of surgery.

[33]  G. James,et al.  The use of standardized patients to evaluate family medicine resident decision making. , 2007, Family medicine.

[34]  B. Charlin,et al.  Is the script concordance test a valid instrument for assessment of intraoperative decision-making skills? , 2007, American journal of surgery.

[35]  R. Wasserman,et al.  Does Clinical Presentation Explain Practice Variability in the Treatment of Febrile Infants? , 2006, Pediatrics.

[36]  Pat Croskerry,et al.  The theory and practice of clinical decision-making , 2005 .

[37]  K. Eva.  What every teacher needs to know about clinical reasoning , 2005, Medical education.

[38]  K. Roberts.  Management and outcomes of care of fever in early infancy. , 2004, The Journal of pediatrics.

[39]  G. Cumming,et al.  A Primer on the Understanding, Use, and Calculation of Confidence Intervals that are Based on Central and Noncentral Distributions , 2001 .

[40]  M. Gittelman,et al.  Management of febrile infants and children by pediatric emergency medicine and emergency medicine: Comparison with practice guidelines , 2001, Pediatric emergency care.

[41]  C. V. D. van der Vleuten,et al.  The Script Concordance Test: A Tool to Assess the Reflective Clinician , 2000, Teaching and learning in medicine.

[42]  J. W. Bass,et al.  A survey about management of febrile children without source by primary care physicians. , 1998, The Pediatric infectious disease journal.

[43]  N. Schmitt.  Uses and abuses of coefficient alpha. , 1996 .

[44]  P. Young.  The management of febrile infants by primary-care pediatricians in Utah: comparison with published practice guidelines. , 1995, Pediatrics.

[45]  E. Shapiro,et al.  Risk factors for development of bacterial meningitis among children with occult bacteremia. , 1986, The Journal of pediatrics.