What is an optimal sequential OSCE model?

Dear Sir We read with great interest the article written by Currie et al. (2015), describing their novel approach to introduce the ‘‘ideal’’ design for sequential objective structured clinical examination (OSCE). The authors have profoundly explored this issue by obtaining data from two large-scale OSCEs, which confirms reproducibility of the findings, and have conducted a statistical modelling which simulated numerous hypothetical screening OSCEs. While we identify this study as innovative, we think that there are some areas for improvement. Our first concern is the rationale for the number of selected stations. The screening tests are supposed to be small enough to effectively reduce the costs. A screening OSCE consisting of 13 stations (for a 15-station OSCE) was excluded by the authors themselves at the end of the study, despite its high specificity and sensitivity; a fact that questions the logic for including so many stations in the model. We believe that considering a range of 6–10 stations could have been more reasonable. Besides, it should be noted that the stations were selected on a random basis, while evidence suggest that stations’ characteristics such as difficulty or discrimination indices might influence the outcomes (Smee et al. 2003). Hence, for a better prediction, it would be advisable to select the screening stations according to their psychometric properties. On the other hand, when evaluating the performance of screening OSCEs, it seems that the looked-for outcomes were not set a priori, and choosing the eight-station OSCE was based on a post hoc decision, considered intuitionally after observing the results. Furthermore, in a condition that only the test result is available and the ultimate outcomes for the examinees are unknown, sensitivity and specificity do not much apply, and stating the ‘‘predictive value’’ of a positive or negative result would be more informative. Moreover, these outcomes are not equally important. Excluding false positives is more essential than eliminating false negatives in sequential testing. Finally, there are other important parameters, such as screening failure rate (known as saving) in a screening OSCE. Therefore, our recommendation is the use of a composite weighted outcome.