Comparing PDA- and paper-based evaluation of the clinical skills of third-year students.

BACKGROUND The mini-Clinical Evaluation Exercise (mini-CEX) is used to assess medical students' clinical skills during medicine clerkships. PURPOSE To evaluate reliability, feasibility, and user satisfaction of a paper vs PDA-based mini-CEX in a third-year medicine clerkship. METHODS The mini-CEX was reformatted as a PDA-based rating form for a medicine clerkship over 1 year. Faculty and residents were instructed to use either paper-based or the PDA form to assess clinical skills of students. A 9-point Likert scale was used to assess clinical skills and user satisfaction. Independent t-tests were used to assess differences between delivery formats. FINDINGS Nearly all (98%) students completed 2 Mini-CEXs, with 275 PDA- and 101 paper-based records performed. Form reliability (Cronbach alpha) exceeded 0.9 for both. Overall resident satisfaction scores with the PDA form (7.2 +/- 1.8) were higher (P = 0.01) than the paper-based form (6.6/1.7). However, faculty satisfaction scores with the PDA form (6.9 +/- 1.6) were significantly lower (P = 0.01) than the paper form (7.6 +/- 1.5). Mean scores for all 7 clinical competencies of PDA format (7.9 +/- 0.9) were higher than the paper-based (7.6 +/- 1.1) version (P = .01). Mean observation (26 min; +/- 16) and feedback time (11 min +/- 8) were longer (both P < .05) with PDA-based form compared to the paper version (22 min +/- 14);(8.7 +/- 6.3). Student and evaluator satisfaction ratings were not significantly different by form. CONCLUSIONS Both PDA- and paper-based mini-CEX delivery was acceptable to evaluators and students with both formats demonstrating high reliability. However, because evaluators' satisfaction, observation, and feedback time differed by form, further studies are needed to determine factors influencing rating variability.