The Associations Between Clerkship Objective Structured Clinical Examination (OSCE) Grades and Subsequent Performance

ABSTRACT Construct: We investigated the extent of the associations between medical students' clinical competency measured by performance in Objective Structured Clinical Examinations (OSCE) during Obstetrics/Gynecology and Family Medicine clerkships and later performance in both undergraduate and graduate medical education. Background: There is a relative dearth of studies on the correlations between undergraduate OSCE scores and future exam performance within either undergraduate or graduate medical education and almost none on linking these simulated encounters to eventual patient care. Of the research studies that do correlate clerkship OSCE scores with future performance, these often have a small sample size and/or include only 1 clerkship. Approach: Students in USU graduating classes of 2007 through 2011 participated in the study. We investigated correlations between clerkship OSCE grades with United States Medical Licensing Examination Step 2 Clinical Knowledge, Clinical Skills, and Step 3 Exams scores as well as Postgraduate Year 1 program director's evaluation scores on Medical Expertise and Professionalism. We also conducted contingency table analysis to examine the associations between poor performance on clerkship OSCEs with failing Step 3 and receiving poor program director ratings. Results: The correlation coefficients were weak between the clerkship OSCE grades and the outcomes. The strongest correlations existed between the clerkship OSCE grades and the Step 2 CS Integrated Clinical Encounter component score, Step 2 Clinical Skills, and Step 3 scores. Contingency table associations between poor performances on both clerkships OSCEs and poor Postgraduate Year 1 Program Director ratings were significant. Conclusions: The results of this study provide additional but limited validity evidence for the use of OSCEs during clinical clerkships given their associations with subsequent performance measures.

[1]  S. Durning,et al.  Validity Evidence for Medical School OSCEs: Associations With USMLE® Step Assessments , 2014, Teaching and learning in medicine.

[2]  D. Ander,et al.  A core competency-based objective structured clinical examination (OSCE) can predict future resident performance. , 2010, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[3]  B. Jolly,et al.  Predictive validity and estimated cut score of an objective structured clinical examination (OSCE) used as an assessment of clinical skills at the end of the first clinical year , 2002, Medical education.

[4]  J. Norcini,et al.  STATUS OF STANDARDIZED PATIENT ASSESSMENT: Methodological Issues in the Use of Standardized Patients for Assessment , 2003, Teaching and learning in medicine.

[5]  Timothy J. Wood,et al.  A procedural skills OSCE: assessing technical and non-technical skills of internal medicine residents , 2015, Advances in health sciences education : theory and practice.

[6]  J. Shatzer,et al.  For Personal Use Only. Reproduce with Permission from the Lancet Publishing Group. Standard Setting Assessment of Clinical Competence Medical Education Quartet , 2022 .

[7]  R. Thomas,et al.  Family medicine resident OSCEs: a systematic review , 2016, Education for primary care : an official publication of the Association of Course Organisers, National Association of GP Tutors, World Organisation of Family Doctors.

[8]  W. Rifkin,et al.  Correlation between housestaff performance on the United States Medical Licensing Examination and standardized patient encounters. , 2005, The Mount Sinai journal of medicine, New York.

[9]  S. Durning,et al.  Correlation of National Board of Medical Examiners Scores with United States Medical Licensing Examination Step 1 and Step 2 Scores , 2012, Academic medicine : journal of the Association of American Medical Colleges.

[10]  D. Ander,et al.  Objective Structured Clinical Examinations Provide Valid Clinical Skills Assessment in Emergency Medicine Education , 2014, The western journal of emergency medicine.

[11]  S. Myung,et al.  Could clinical experience during clerkship enhance students’ clinical performance? , 2014, BMC Medical Education.

[12]  H. Loeser,et al.  Early Detection and Evaluation of Professionalism Deficiencies in Medical Students: One School's Approach , 2001, Academic medicine : journal of the Association of American Medical Colleges.

[13]  S. Durning,et al.  40 years of military medical education: an overview of the Long-Term Career Outcome Study (LTCOS). , 2012, Military medicine.

[14]  J. Colliver,et al.  Validating the standardized‐patient assessment administered to medical students in the New York City Consortium , 1997, Academic medicine : journal of the Association of American Medical Colleges.

[15]  T. Lesnick,et al.  Validity Evidence for an OSCE to Assess Competency in Systems-Based Practice and Practice-Based Learning and Improvement: A Preliminary Investigation , 2008, Academic medicine : journal of the Association of American Medical Colleges.

[16]  S. Durning,et al.  The Feasibility, Reliability, and Validity of a Program Director’s (Supervisor’s) Evaluation Form for Medical School Graduates , 2005, Academic medicine : journal of the Association of American Medical Colleges.

[17]  Michael T. Kane,et al.  An argument-based approach to validity. , 1992 .

[18]  Winny Setyonugroho,et al.  Reliability and validity of OSCE checklists used to assess the communication skills of undergraduate medical students: A systematic review. , 2015, Patient education and counseling.

[19]  S. Durning,et al.  Development and initial validation of a program director's evaluation form for medical school graduates. , 2015, Military medicine.

[20]  M. Härter,et al.  Assessing Communication Skills of Medical Students in Objective Structured Clinical Examinations (OSCE) - A Systematic Review of Rating Scales , 2016, PloS one.