Assessment of clinical skills with standardized patients: State of the art

A little more than 10 years ago, the objective structured clinical examination (OSCE) was introduced. It includes several “stations,”; at which examinees perform a variety of clinical tasks. Although an OSCE may involve a range of testing methods, standardized patients (SPs), who are nonphysicians trained to play the role of a patient, are commonly used to assess clinical skills. This article provides a comprehensive review of large‐scale studies of the psychometric characteristics of SP‐based tests. Across studies, reliability analyses consistently indicate that the major source of measurement error is variation in examinee performance from station to station (termed content specificity in the medical‐problem‐solving literature). As a consequence, tests must include large numbers of stations to obtain a stable, reproducible assessment of examinee skills. Disagreements among raters observing examinee performance and differences between SPs playing the same patient role appear to have less effect on the pr...

[1]  A. Owen,et al.  GENERAL PRACTITIONERS AND PSYCHOSOCIAL PROBLEMS AN EVALUATION USING PSEUDOPATIENTS , 1974 .

[2]  M. B. Regan,et al.  An assessment of the clinical skills of fourth‐year students at four New England medical schools , 1990, Academic medicine : journal of the Association of American Medical Colleges.

[3]  J. Norcini,et al.  Factors influencing reproducibility of tests using standardized patients , 1989 .

[4]  J. Maatsch,et al.  An evaluation of the construct validity of four alternative theories of clinical competence. , 1986, Research in medical education : proceedings of the ... annual Conference. Conference on Research in Medical Education.

[5]  M. B. Regan,et al.  A diagnostic fourth-year performance assessment. , 1987, Archives of internal medicine.

[6]  D. Steward,et al.  An assessment of the consistency and accuracy of standardized patients' simulations. , 1987, Journal of medical education.

[7]  Reed G. Williams,et al.  Reliability of performance on standardized patient cases: A comparison of consistency measures based on generalizability theory , 1989 .

[8]  R. Tamblyn,et al.  How standardized are standardized patients? , 1988, Research in medical education : proceedings of the ... annual Conference. Conference on Research in Medical Education.

[9]  R. Harden,et al.  Assessment of clinical competence using objective structured examination. , 1975, British medical journal.

[10]  J. Rethans,et al.  Simulated patients in general practice: a different look at the consultation. , 1987, British medical journal.

[11]  D. Newble,et al.  The effect of assessments and examinations on the learning of medical students , 1983, Medical education.

[12]  C. Vleuten,et al.  Training and experience of examiners , 1989, Medical education.

[13]  E. Graaff,et al.  Validation of a new measure of clinical problem‐solving , 1987, Medical education.

[14]  P. Stillman,et al.  Patient instructors as teachers and evaluators. , 1980, Journal of medical education.

[15]  E. Petrusa,et al.  An objective measure of clinical performance. , 1987, The American journal of medicine.

[16]  A. Stillman,et al.  Assessing clinical skills of residents with standardized patients. , 1986, Annals of internal medicine.

[17]  D. Newble,et al.  The validity and reliability of a new examination of the clinical competence of medical students , 1981, Medical education.

[18]  C. Vleuten,et al.  A written test as an alternative to performance testing , 1989, Medical education.

[19]  D. Swanson,et al.  Ensuring the clinical competence of medical school graduates through standardized patients. , 1987, Archives of internal medicine.

[20]  D. Newble,et al.  The selection and training of examiners for clinical examinations , 1980, Medical education.

[21]  L. Shulman,et al.  Medical problem-solving. , 2014, Journal of medical education.

[22]  H. Barrows,et al.  A comprehensive performance-based assessment of fourth-year students' clinical skills. , 1987, Journal of medical education.

[23]  Robert L. Ebel,et al.  Must all tests be valid , 1961 .

[24]  M. J. Kolen.  Traditional Equating Methodology , 1988 .

[25]  P Tugwell,et al.  A comparison of resident performance on real and simulated patients. , 1982, Journal of medical education.

[26]  R. Dawes,et al.  Linear models in decision making. , 1974 .

[27]  B. J. Andrew.  The use of behavioral checklists to assess physical examination skills. , 1977, Journal of medical education.

[28]  J. Beauchemin,et al.  Practice settings and prescribing profiles: the simulation of tension headaches to general practitioners working in different practice settings in the Montreal area. , 1980, American journal of public health.

[29]  D. Newble.  Eight years' experience with a structured clinical examination , 1988, Medical education.

[30]  L. Perkowski,et al.  A multiple station objective clinical evaluation. , 1984, Research in medical education : proceedings of the ... annual Conference. Conference on Research in Medical Education.

[31]  H. Barrows,et al.  Direct, standardized assessment of clinical competence , 1987, Medical education.

[32]  M. Ray Karnes,et al.  Measuring educational achievement , 1950 .

[33]  C. Woodward,et al.  Measuring physicians' performances by using simulated patients. , 1985, Journal of medical education.

[34]  H. Conn,et al.  Results of the second Clinical Skills Assessment Examination of the ECFMG , 1989, Academic medicine : journal of the Association of American Medical Colleges.

[35]  H. Conn.  Assessing the clinical skills of foreign medical graduates. , 1986, Journal of medical education.

[36]  J. Norcini,et al.  Factors influencing testing time requirements for measurements using written simulations , 1989 .

[37]  K. Hayes.  A Comprehensive Performance-based Assessment of Fourth-year Students’ Clinical Skills. Barrows HS, Williams RG, Moy RH (Southern Illinois University School of Medicine, Springfield, IL), Journal of Medical Education 62:805-809, 1987. , 1988 .

[38]  N. Frederiksen.  The real test bias: Influences of testing on teaching and learning. , 1984 .

[39]  J. Frederiksen,et al.  A Systems Approach to Educational Testing , 1989 .

[40]  J. Ludbrook,et al.  Examiner training for clinical examinations , 1971, British journal of medical education.

[41]  D. Newble,et al.  Psychometric characteristics of the objective structured clinical examination , 1988, Medical education.

[42]  C. V. D. van der Vleuten,et al.  The OSCE as a part of a systematic skills training approach. , 1987, Medical teacher.

[43]  Noel Entwistle,et al.  Styles of learning and teaching , 1981 .

[44]  D. Newble,et al.  A problem-based criterion-referenced examination of clinical competence. , 1978, Journal of medical education.

[45]  B. Templeton,et al.  The Relationships of Pediatric Resident Recording Behavior Across Medical Conditions , 1980, Medical care.

[46]  R. Brennan.  Elements of generalizability theory , 1983 .

[47]  R W Sanson-Fisher,et al.  Simulated patients and the assessment of medical students' interpersonal skills , 1980, Medical education.

[48]  David B. Swanson,et al.  Assessment of Clinical Competence: Written and Computer-Based Simulations , 1987 .

[49]  Michael T. Kane,et al.  The validity of licensure examinations. , 1982 .