A unified approach to validation, reliability, and education study design for surgical technical skills training.

OBJECTIVE To present modern educational psychology theory and apply these concepts to validity and reliability of surgical skills training and assessment. DESIGN In a series of cross-disciplinary meetings, we applied a unified approach of behavioral science principles and theory to medical technical skills education given the recent advances in the theories in the field of behavioral psychology and statistics. CONCLUSIONS While validation of the individual simulation tools is important, it is only one piece of a multimodal curriculum that in and of itself deserves examination and study. We propose concurrent validation throughout the design of simulation-based curriculum rather than once it is complete. We embrace the concept that validity and curriculum development are interdependent, ongoing processes that are never truly complete. Individual predictive, construct, content, and face validity aspects should not be considered separately but as interdependent and complementary toward an end application. Such an approach could help guide our acceptance and appropriate application of these exciting new training and assessment tools for technical skills training in medicine.

[1]  S. Messick Meaning and Values in Test Validation: The Science and Ethics of Assessment , 1989 .

[2]  R. Satava,et al.  Virtual Reality Training Improves Operating Room Performance: Results of a Randomized, Double-Blinded Study , 2002, Annals of surgery.

[3]  S. Downing Validity: on the meaningful interpretation of assessment data , 2003, Medical education.

[4]  Amy Brualdi,et al.  Traditional and Modern Concepts of Validity. ERIC/AE Digest. , 1999 .

[5]  A. K. Sachdeva Invited commentary: Educational interventions to address the core competencies in surgery. , 2004, Surgery.

[6]  R. Reznick,et al.  Testing technical skill via an innovative "bench station" examination. , 1997, American journal of surgery.

[7]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[8]  G. Wiggins,et al.  Understanding by Design , 1998 .

[9]  Richard M Satava,et al.  Virtual reality surgical simulator: the first steps. 1993. , 2006, Clinical orthopaedics and related research.

[10]  M. Kane Current Concerns in Validity Theory , 2001 .

[11]  G. Fried,et al.  Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. , 2004, Surgery.

[12]  P. Moss Themes and Variations in Validity Theory , 2005 .

[13]  Gilbert Sax,et al.  Principles of educational and psychological measurement and evaluation , 1980 .

[14]  E. Simpson THE CLASSIFICATION OF EDUCATIONAL OBJECTIVES, PSYCHOMOTOR DOMAIN. , 1966 .

[15]  S. Messick Validity of Psychological Assessment: Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning. Research Report RR-94-45. , 1994 .

[16]  Samuel Messick,et al.  VALIDITY OF TEST INTERPRETATION AND USE , 1990 .

[17]  David Prideaux,et al.  Curriculum design , 2003, BMJ : British Medical Journal.

[18]  R. M. Satava,et al.  Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training , 2003, Surgical Endoscopy And Other Interventional Techniques.

[19]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .