Simulation-based assessments in health professional education: a systematic review

Introduction The use of simulation in health professional education has increased rapidly over the past 2 decades. While simulation has predominantly been used to train health professionals and students for a variety of clinically related situations, there is an increasing trend to use simulation as an assessment tool, especially for the development of technical-based skills required during clinical practice. However, there is a lack of evidence about the effectiveness of using simulation for the assessment of competency. Therefore, the aim of this systematic review was to examine simulation as an assessment tool of technical skills across health professional education. Methods A systematic review of Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), Medical Literature Analysis and Retrieval System Online (Medline), and Web of Science databases was used to identify research studies published in English between 2000 and 2015 reporting on measures of validity, reliability, or feasibility of simulation as an assessment tool. The McMasters Critical Review for quantitative studies was used to determine methodological value on all full-text reviewed articles. Simulation techniques using human patient simulators, standardized patients, task trainers, and virtual reality were included. Results A total of 1,064 articles were identified using search criteria, and 67 full-text articles were screened for eligibility. Twenty-one articles were included in the final review. The findings indicated that simulation was more robust when used as an assessment in combination with other assessment tools and when more than one simulation scenario was used. Limitations of the research papers included small participant numbers, poor methodological quality, and predominance of studies from medicine, which preclude any definite conclusions. Conclusion Simulation has now been embedded across a range of health professional education and it appears that simulation-based assessments can be used effectively. However, the effectiveness as a stand-alone assessment tool requires further research.

[1]  Guillaume Alinier,et al.  Nursing students' and lecturers' perspectives of objective structured clinical examination incorporating simulation. , 2003, Nurse education today.

[2]  Felicity Blackstock,et al.  Can simulation replace part of clinical time? Two parallel randomised controlled trials , 2012, Medical education.

[3]  N. Falchikov,et al.  Student Peer Assessment in Higher Education: A Meta-Analysis Comparing Peer and Teacher Marks , 2000 .

[4]  W. Ummenhofer,et al.  Technical and non‐technical skills can be reliably assessed during paramedic simulation training , 2009, Acta anaesthesiologica Scandinavica.

[5]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[6]  M. J. Gordon A review of the validity and accuracy of self‐assessments in health professions training , 1991, Academic medicine : journal of the Association of American Medical Colleges.

[7]  D. Gaba The future vision of simulation in health care , 2004, Quality and Safety in Health Care.

[8]  Fiona Paul An exploration of student nurses' thoughts and experiences of using a video-recording to assess their performance of cardiopulmonary resuscitation (CPR) during a mock objective structured clinical examination (OSCE). , 2010, Nurse education in practice.

[9]  Teodor P. Grantcharov,et al.  Objective Assessment of Gastrointestinal Endoscopy Skills Using a Virtual Reality Simulator , 2005, JSLS : Journal of the Society of Laparoendoscopic Surgeons.

[10]  H. Rushforth,et al.  Objective structured clinical examination (OSCE): review of literature and implications for nursing education. , 2007, Nurse education today.

[11]  Mandy J Brimble Skills assessment using video analysis in a simulated environment: an evaluation. , 2008, Paediatric nursing.

[12]  Use of a modified OSCE to assess nurse practitioner students. , 2008, British journal of nursing.

[13]  I. Marzi,et al.  Increased authenticity in practical assessment using emergency case OSCE stations , 2010, Advances in health sciences education : theory and practice.

[14]  Rajesh Aggarwal,et al.  An Evidence-Based Virtual Reality Training Program for Novice Laparoscopic Surgeons , 2006, Annals of surgery.

[15]  A. Errichetti,et al.  Evaluating the clinical skills of osteopathic medical students , 2003, The Journal of the American Osteopathic Association.

[16]  G. Regehr,et al.  The Psychiatry OSCE: A 20-Year Retrospective , 2014, Academic Psychiatry.

[17]  Kathleen R Rosen,et al.  The history of medical simulation. , 2008, Journal of critical care.

[18]  D. Boud Sustainable Assessment: Rethinking assessment for the learning society , 2000 .

[19]  L. Sheppard,et al.  The patient experience in the emergency department: A systematic synthesis of qualitative research. , 2010, International emergency nursing.

[20]  Geoffrey R. Norman,et al.  Performance-Based Assessment: Lessons From the Health Professions , 1995 .

[21]  G. Wiggins A True Test: Toward More Authentic and Equitable Assessment , 2011 .

[22]  G. Norman,et al.  A comprehensive checklist for reporting the use of OSCEs , 2009, Medical teacher.

[23]  B. Egan,et al.  Designing and Implementing the Objective Structured Clinical Examination in Anesthesiology , 2014, Anesthesiology.

[24]  John R Boulet,et al.  Performance of Residents and Anesthesiologists in a Simulation-based Skill Assessment , 2007, Anesthesiology.

[25]  I. Koren,et al.  Objective structured clinical evaluation of clinical competence: an integrative review. , 2009, Journal of advanced nursing.

[26]  S. Schabrun,et al.  Healthcare equipment as a source of nosocomial infection: a systematic review. , 2006, The Journal of hospital infection.

[27]  R. Watson,et al.  Clinical competence assessment in nursing: a systematic review of the literature. , 2002, Journal of advanced nursing.

[28]  M. Julião,et al.  Is the OSCE a feasible tool to assess competencies in undergraduate medical education? , 2013, Medical teacher.

[29]  S. Barry Issenberg,et al.  Simulation in Health Care Education , 2008, Perspectives in biology and medicine.

[30]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[31]  C. Carraccio,et al.  The objective structured clinical examination: a step in the direction of competency-based evaluation. , 2000, Archives of pediatrics & adolescent medicine.

[32]  K. Grimmer,et al.  No model of clinical education for physiotherapy students is superior to another: a systematic review. , 2007, The Australian journal of physiotherapy.

[33]  R. Lammers,et al.  Simulation-based Assessment of Paramedic Pediatric Resuscitation Skills , 2007 .

[34]  Guillaume Alinier,et al.  Determining the value of simulation in nurse education: study design and initial results. , 2004, Nurse education in practice.

[35]  Michael A Seropian,et al.  Simulation: not just a manikin. , 2004, The Journal of nursing education.

[36]  J. Zuckerman,et al.  Objective Structured Clinical Examinations: A Guide to Development and Implementation in Orthopaedic Residency , 2013, The Journal of the American Academy of Orthopaedic Surgeons.

[37]  Karen J. Panzarella,et al.  Using the Integrated Standardized Patient Examination to Assess Clinical Competence in Physical Therapist Students , 2008 .

[38]  John R Boulet,et al.  Simulation in Pediatrics: The Reliability and Validity of a Multiscenario Assessment , 2011, Pediatrics.

[39]  E. Madigan,et al.  Using low-fidelity simulation with sophomore nursing students in a baccalaureate nursing program. , 2012, Nursing education perspectives.

[40]  W. Bartfay,et al.  Evaluation. The OSCE approach in nursing education. , 2004, The Canadian nurse.

[41]  R. Rao,et al.  SIMULATED PATIENTS AND OBJECTIVE STRUCTURED CLINICAL EXAMINATIONS: REVIEW OF THEIR USE IN MEDICAL EDUCATION , 2002 .

[42]  Petra M. Casey,et al.  To the point: reviews in medical education--the Objective Structured Clinical Examination. , 2009, American journal of obstetrics and gynecology.

[43]  G. Bergus,et al.  Comparison of Medical Student and Physician Assistant Student Performance on Standardized‐Patient Assessments , 2007 .

[44]  C. Pearsall,et al.  Undergraduate Students as Standardized Patients to Assess Advanced Practice Nursing Student Competencies , 2009, Nurse educator.

[45]  J. Messenger,et al.  A Technical and Cognitive Skills Evaluation of Performance in Interventional Cardiology Procedures Using Medical Simulation , 2010, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[46]  A. Ziv,et al.  Simulation-based medical education: an ethical imperative. , 2006, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[47]  A. Barman Critiques on the Objective Structured Clinical Examination. , 2005, Annals of the Academy of Medicine, Singapore.

[48]  M. Egan,et al.  Dementia and Occupation: A Review of the Literature , 2006, Canadian journal of occupational therapy. Revue canadienne d'ergotherapie.

[49]  Valerie Smith,et al.  The Objective Structured Clinical Examination (OSCE) as a strategy for assessing clinical competence in midwifery education in Ireland: a critical review. , 2012, Nurse education in practice.

[50]  Amitai Ziv,et al.  Patient safety and simulation-based medical education , 2000, Medical teacher.

[51]  David J. Murray,et al.  Management of Anesthesia Equipment Failure: A Simulation-Based Resident Skill Assessment , 2009, Anesthesia and analgesia.

[52]  André De Champlain,et al.  Assessment of patient management skills and clinical skills of practising doctors using computer‐based case simulations and standardised patients , 2004, Medical education.

[53]  M. Dankoski,et al.  Objective structured clinical exams: a critical review. , 2008, Family medicine.

[54]  D. Murray,et al.  Simulation-based assessment of pediatric anesthesia skills. , 2011, Anesthesiology.

[55]  D. Murray,et al.  Acute care of pediatric patients with sickle cell disease: A simulation performance assessment , 2013, Pediatric blood & cancer.

[56]  K. Masaki,et al.  Using Standardized Patients to Assess the Geriatrics Medicine Skills of Medical Students, Internal Medicine Residents, and Geriatrics Medicine Fellows , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[57]  Richard Fuller,et al.  How to measure the quality of the OSCE: A review of metrics – AMEE guide no. 49 , 2010, Medical teacher.

[58]  D. Nestel,et al.  Simulation in clinical teaching and learning , 2012, The Medical journal of Australia.

[59]  Roger Watson,et al.  Assessing competence to practise in nursing: a review of the literature , 2002 .

[60]  Uno Fors,et al.  Virtual patients for assessment of clinical reasoning in nursing -- a pilot study. , 2011, Nurse education today.

[61]  D. Irby,et al.  Assessment in medical education. , 2007, The New England journal of medicine.

[62]  H. Connors,et al.  STANDARDIZED PATIENT EXPERIENCES: Evaluation of Clinical Performance and Nurse Practitioner Student Satisfaction , 2004, Nursing education perspectives.

[63]  A. Udy,et al.  A Prospective Comparison between Written Examination and Either Simulation-Based or Oral Viva Examination of Intensive Care Trainees’ Procedural Skills , 2010, Anaesthesia and intensive care.

[64]  K. Parakh Assessment in medical education. , 2007, The New England journal of medicine.

[65]  M. Mitchell,et al.  The objective structured clinical examination (OSCE): optimising its value in the undergraduate nursing curriculum. , 2009, Nurse education today.

[66]  Raymond F. Peterson,et al.  Simulation Can Contribute a Part of Cardiorespiratory Physiotherapy Clinical Education: Two Randomized Trials , 2013, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[67]  Margaret M. Plack,et al.  Validating a Standardized Patient Assessment Tool Using Published Professional Standards , 2011 .

[68]  Lesley Mole,et al.  Early introduction to interprofessional learning: a simulated ward environment , 2003, Medical education.

[69]  Chad E. Wagner,et al.  Comparison of Expert and Novice Performance of a Simulated Transesophageal Echocardiography Examination , 2013, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[70]  B. Hodges Validity and the OSCE , 2003, Medical Teacher.

[71]  B. Penprase,et al.  The use of high-fidelity simulation in the admissions process: one nurse anesthesia program's experience. , 2012, AANA journal.

[72]  M. Wilkes,et al.  A Comparative Study of Measures to Evaluate Medical Students' Performances , 2000, Academic medicine : journal of the Association of American Medical Colleges.

[73]  Matthew S. Prewett,et al.  A systematic review of the reliability of objective structured clinical examination scores , 2011, Medical education.

[74]  David Murray,et al.  Reliability and Validity of a Simulation-based Acute Care Skills Assessment for Medical Students and Residents , 2003, Anesthesiology.

[75]  Michael Haubner,et al.  Virtual reality in medicine-computer graphics and interaction techniques , 1997, IEEE Transactions on Information Technology in Biomedicine.

[76]  A. Greiner,et al.  Health Professions Education: A Bridge to Quality , 2003 .

[77]  Silvio H. R. Rizzi,et al.  Virtual reality simulations. , 2007, Anesthesiology clinics.

[78]  J. McAllister,et al.  Assessing the validity evidence of an objective structured assessment tool of technical skills for neonatal lumbar punctures. , 2013, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[79]  C. Peterson Health professions education: a bridge to quality. , 2003, Tar heel nurse.

[80]  J. Weller,et al.  Psychometric characteristics of simulation‐based assessment in anaesthesia and accuracy of self‐assessed scores * , 2005, Anaesthesia.