Comparison of Two Standard-setting Methods for Advanced Cardiac Life Support Training

Background This study used the Angoff and Hofstee standard-setting methods to derive minimum passing scores for six advanced cardiac life support (ACLS) procedures. Method An expert panel provided item-based (Angoff) and group-based (Hofstee) judgments about six ACLS performance checklists on two occasions separated by ten weeks. Interrater reliabilities and test-retest reliability (stability) of the judgments were calculated. Derived ACLS passing standards are compared to historical ACLS performance data from two groups of ACLS-trained internal medicine residents. Results Both the Angoff and Hofstee standard-setting methods produced reliable and stable data. Hofstee minimum passing scores (MPSs) were uniformly more stringent than Angoff MPSs. Interpretation of historical ACLS performance data from medical residents shows the MPSs derived in this study would yield higher-than-expected failure rates. Conclusion Systematic standard setting for ACLS procedures is a necessary step toward the creation of mastery learning educational programs.

[1]  M. Cusimano,et al.  The Effect of Incorporating Normative Data into a Criterion-Referenced Standard Setting in Medical Education , 2003, Academic medicine : journal of the Association of American Medical Colleges.

[2]  P. Steen,et al.  Quality of cardiopulmonary resuscitation during out-of-hospital cardiac arrest. , 2005, JAMA.

[3]  M. Kane.  Validating the Performance Standards Associated With Passing Scores , 1994 .

[4]  J. DeVries,et al.  Evaluation of clinical competence: the gap between expectation and performance. , 1996, Pediatrics.

[5]  Rebecca S. Lipner,et al.  A Comparison of Three Variations on a Standard-Setting Method. , 1987 .

[6]  Diana H. J. M. Dolmans,et al.  International handbook of research in medical education , 2002 .

[7]  J. Feinglass,et al.  Graduating internal medicine residents’ self-assessment and performance of advanced cardiac life support skills , 2006, Medical teacher.

[8]  M. Cusimano,et al.  Consistency of Standards and Stability of Pass/Fail Decisions with Examinee-Based Standard-Setting Methods in a Small-Scale Objective Structured Clinical Examination , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[9]  J. Shea,et al.  The Reproducibility of Standards Over Groups and Occasions , 1992 .

[10]  William C. McGaghie,et al.  Simulation-Based Training of Internal Medicine Residents in Advanced Cardiac Life Support Protocols: A Randomized Trial , 2005, Teaching and learning in medicine.

[11]  L. Nieman,et al.  Cardiac auscultatory skills of internal medicine and family practice trainees. A comparison of diagnostic proficiency. , 1997, JAMA.

[12]  J. Norcini,et al.  Combining Tests and Setting Standards , 2002 .

[13]  W. McGaghie,et al.  Competency-Based Curriculum Development in Medical Education , 1978 .

[14]  K. A. Ericsson,et al.  Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[15]  A W Sajid,et al.  Competency-based curriculum development on medical education: an introduction. , 1978, Public health papers.

[16]  M. Cusimano,et al.  Defining standards of competent performance on an OSCE. , 1998, Academic medicine : journal of the Association of American Medical Colleges.

[17]  S. Kilminster,et al.  Standard Setting for OSCEs: Trial of Borderline Approach , 2004, Advances in health sciences education : theory and practice.

[18]  Dato N. M. de Gruijter,et al.  COMPROMISE MODELS FOR ESTABLISHING EXAMINATION STANDARDS , 1985 .

[19]  M. Hazinski,et al.  In-hospital resuscitation: a statement for healthcare professionals from the American Heart Association Emergency Cardiac Care Committee and the Advanced Cardiac Life Support, Basic Life Support, Pediatric Resuscitation, and Program Administration Subcommittees. , 1997, Circulation.

[20]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[21]  Lars Wik,et al.  Retention of basic life support skills 6 months after training with an automated voice advisory manikin system without instructor involvement. , 2002, Resuscitation.

[22]  M. Cusimano.  Standard setting in medical education , 1996, Academic medicine : journal of the Association of American Medical Colleges.