A Blueprint to Assess Professionalism: Results of a Systematic Review

Purpose Assessing professionalism is hampered by varying definitions and these definitions' lack of a clear breakdown of the elements of professionalism into aspects that can be measured. Professionalism is multidimensional, so a combination of assessment tools is required. In this study, conducted during 2007–2008, the authors aimed to match assessment tools to definable elements of professionalism and to identify gaps where professionalism elements are not well addressed by existing assessment tools. Method The authors conducted literature reviews of definitions of professionalism and of relevant assessment tools, clustered the definitions of professionalism into assessable components, and clustered assessment tools of a similar nature. They then created a “blueprint” whereby the elements of professionalism are matched to relevant assessment tools. Results Five clusters of professionalism were formed: adherence to ethical practice principles, effective interactions with patients and with people who are important to those patients, effective interactions with people working within the health system, reliability, and commitment to autonomous maintenance / improvement of competence in oneself, others, and systems. Nine clusters of assessment tools were identified: observed clinical encounters, collated views of coworkers, records of incidents of unprofessionalism, critical incident reports, simulations, paper-based tests, patients' opinions, global views of supervisor, and self-administered rating scales. Conclusions Professionalism can be assessed using a combination of observed clinical encounters, multisource feedback, patients' opinions, paper-based tests or simulations, measures of research and/or teaching activities, and scrutiny of self-assessments compared with assessments by others. Attributes that require more development in their measurement are reflectiveness, advocacy, lifelong learning, dealing with uncertainty, balancing availability to others with care for oneself, and seeking and responding to results of an audit.

[1]  J. Norcini,et al.  Peer assessment of competence , 2003, Medical education.

[2]  Mary Gleason Heffron,et al.  Objective Structured Video Examinations (OSVEs) for Geriatrics Education , 2006, Gerontology & geriatrics education.

[3]  H. Slotnick,et al.  Proto‐professionalism: how professionalisation occurs across the continuum of medical education , 2005, Medical education.

[4]  C. Violato,et al.  The assessment of emergency physicians by a regulatory authority. , 2006, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[5]  H. Swick Toward a Normative Definition of Medical Professionalism , 2000, Academic medicine : journal of the Association of American Medical Colleges.

[6]  J. Medalie,et al.  Patient perceptions of humanism in physicians: effects on positive health behaviors. , 1990, Family medicine.

[7]  Kelly A Fryer-Edwards,et al.  Promoting Reflection on Professionalism: A Comparison Trial of Educational Interventions for Medical Students , 2003, Academic medicine : journal of the Association of American Medical Colleges.

[8]  S. Durning,et al.  Assessing the Reliability and Validity of the Mini—Clinical Evaluation Exercise for Internal Medicine Residency Training , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[9]  M. Abrahamowicz,et al.  The feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents , 1994, Journal of General Internal Medicine.

[10]  R. K. Allday Report to the Working Party , 1965 .

[11]  W. Bunch,et al.  Moral Reasoning, Professionalism, and the Teaching of Ethics to Orthopaedic Surgeons , 2000, Clinical orthopaedics and related research.

[12]  W. K. Kennedy,et al.  Development of an instrument to measure professionalism. , 2006, American journal of pharmaceutical education.

[13]  S. Melnick,et al.  Professionalism in medical education: The development and validation of a survey instrument to assess attitudes toward professionalism , 2007, Medical teacher.

[14]  K. Szauter,et al.  Medical Student Professionalism: Are We Measuring the Right Behaviors? A Comparison of Professional Lapses by Students and Physicians , 2006, Academic medicine : journal of the Association of American Medical Colleges.

[15]  Michael A Gisondi,et al.  Assessment of resident professionalism using high-fidelity simulation of ethical dilemmas. , 2004, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[16]  A. Yarpuzlu,et al.  Assessment of Time Management Attitudes Among Health Managers , 2005, The health care manager.

[17]  Peggy Wagner,et al.  Defining medical professionalism: a qualitative study , 2007, Medical education.

[18]  T. Wilkinson,et al.  Problems with using a supervisor’s report as a form of summative assessment , 2007, Postgraduate Medical Journal.

[19]  T. Roberts,et al.  Perceptions of professionalism in medicine: a qualitative study , 2006, Medical education.

[20]  A. Kalet,et al.  Promoting Professionalism through an Online Professional Development Portfolio: Successes, Joys, and Frustrations , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[21]  G. Elwyn,et al.  Review of instruments for peer assessment of physicians , 2004, BMJ : British Medical Journal.

[22]  L. Arnold,et al.  Assessing Professional Behavior: Yesterday, Today, and Tomorrow , 2002, Academic medicine : journal of the Association of American Medical Colleges.

[23]  J. Norcini,et al.  The Mini-CEX: A Method for Assessing Clinical Skills , 2003, Annals of Internal Medicine.

[24]  J. Archer,et al.  Use of SPRAT for peer review of paediatricians in training , 2005, BMJ : British Medical Journal.

[25]  F. Camacho,et al.  Measuring Patients’ Trust in their Primary Care Providers , 2002, Medical care research and review : MCRR.

[26]  Jordan J. Cohen Professionalism in medical education, an American perspective: from evidence to accountability , 2006, Medical education.

[27]  D. Lynch,et al.  Assessing professionalism: a review of the literature , 2004, Medical teacher.

[28]  Lorelei Lingard,et al.  Basing the Evaluation of Professionalism on Observable Behaviors: A Cautionary Tale , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[29]  Shiphra Ginsburg,et al.  The Professionalism Mini-Evaluation Exercise: A Preliminary Investigation , 2006, Academic medicine : journal of the Association of American Medical Colleges.

[30]  G. Hughes Understanding doctors: harnessing professionalism , 2008, Emergency Medicine Journal.

[31]  G. Norman,et al.  Performance‐based assessment of clinical ethics using an objective structured clinical examination , 1996, Academic medicine : journal of the Association of American Medical Colleges.

[32]  R. Grol,et al.  Professionalism in general practice: development of an instrument to assess professional behaviour in general practitioner trainees , 2006, Medical education.

[33]  R. Grol,et al.  How to conceptualize professionalism: a qualitative study , 2004, Medical teacher.

[34]  D. Chard,et al.  Medical professionalism: the trainees' views. , 2006, Clinical medicine.

[35]  T. Wilkinson,et al.  Assessment of clinical performance: gathering evidence , 2007, Internal medicine journal.

[36]  Development of a physician attributes database as a resource for medical education, professionalism and student evaluation , 2004, Medical teacher.

[37]  C. Roberts,et al.  Discovering professionalism through guided reflection , 2006, Medical teacher.

[38]  R. Salcido,et al.  Pilot Study of a 360-Degree Assessment Instrument for Physical Medicine & Rehabilitation Residency Programs , 2003, American journal of physical medicine & rehabilitation.

[39]  Medical Professionalism Medical professionalism in the new millennium: a physicians' charter , 2002, The Lancet.

[40]  K. Shaw,et al.  A study of a multi‐source feedback system for international medical graduates holding defined licences , 2006, Medical education.

[41]  F. Oort,et al.  Assessment of medical students' communicative behaviour and attitudes: estimating the reliability of the use of the Amsterdam attitudes and communication scale through generalisability coefficients. , 2001, Patient education and counseling.

[42]  C. Anderson,et al.  Reliability of a core competency checklist assessment in the emergency department: the Standardized Direct Observation Assessment Tool. , 2006, Academic emergency medicine : official journal of the Society for Academic Emergency Medicine.

[43]  J. Savageau,et al.  The effect of a global multiculturalism track on cultural competence of preclinical medical students. , 2001, Family medicine.

[44]  I. Liberzon,et al.  A Cultural Sensitivity Training Workshop for Psychiatry Residents , 2000 .

[45]  Jocelyn Lockyer,et al.  Multisource feedback in the assessment of physician competencies. , 2003, The Journal of continuing education in the health professions.

[46]  S. Kalishman,et al.  Can There Be a Single System for Peer Assessment of Professionalism among Medical Students? A Multi-Institutional Study , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[47]  D. Lie,et al.  Using Standardized Patients to Assess Medical Students' Professionalism , 2001, Academic medicine : journal of the Association of American Medical Colleges.

[48]  L. Gruppen,et al.  Initial use of a novel instrument to measure professionalism in surgical residents. , 2005, American journal of surgery.

[49]  A. Ziv,et al.  Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review , 2005, Medical teacher.

[50]  S. Reddy,et al.  Promoting an environment of professionalism: the University of Chicago "Roadmap". , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[51]  G. Regehr,et al.  OSCE checklists do not capture increasing levels of expertise. , 1999, Academic medicine : journal of the Association of American Medical Colleges.

[52]  B. Majumdar,et al.  Cultural sensitivity training among foreign medical graduates , 1999, Medical education.

[53]  C. Rees,et al.  The acceptability of 360‐degree judgements as a method of assessing undergraduate medical students' personal and professional behaviours , 2005, Medical education.

[54]  T. Wilkinson,et al.  Patients' global ratings of student competence. Unreliable contamination or gold standard? , 2002, Medical Education.

[55]  J. Rademakers,et al.  Development of an instrument to assess professional behaviour of foreign medical graduates , 2007, Medical teacher.

[56]  K. Golnik,et al.  The ophthalmic clinical evaluation exercise: reliability determination. , 2005, Ophthalmology.

[57]  R. Epstein,et al.  Comprehensive Assessment of Professional Competence: The Rochester Experiment , 2004, Teaching and learning in medicine.

[58]  J. Rennie,et al.  The patient assessment questionnaire: A new instrument for evaluating the interpersonal skills of vocational dental practitioners , 2004, British Dental Journal.

[59]  T. Habermann,et al.  A Prospective Study of the Relationship between Medical Knowledge and Professionalism among Internal Medicine Residents , 2007, Academic medicine : journal of the Association of American Medical Colleges.

[60]  E. Alper,et al.  Assessing professionalism in the context of an objective structured clinical examination: an in‐depth study of the rating process , 2007, Medical education.

[61]  F. Oort,et al.  Summative assessment of medical students' communication skills and professional attitudes through observation in clinical practice , 2005, Medical teacher.

[62]  H. Loeser,et al.  Early Detection and Evaluation of Professionalism Deficiencies in Medical Students: One School's Approach , 2001, Academic medicine : journal of the Association of American Medical Colleges.

[63]  D. Stern,et al.  The nature of qualitative comments in evaluating professionalism , 2005, Medical education.

[64]  T. Wilkinson,et al.  Objectivity in Objective Structured Clinical Examinations: Checklists Are No Substitute for Examiner Commitment , 2003, Academic medicine : journal of the Association of American Medical Colleges.

[65]  J. Carline,et al.  Use of peer ratings to evaluate physician performance. , 1993, JAMA.

[66]  John R Boulet,et al.  Using a standardised patient assessment to measure professional attributes , 2005, Medical education.

[67]  David Newble,et al.  The use of portfolios for assessment of the competence and performance of doctors in practice , 2002, Medical education.

[68]  Janke Cohen-Schotanus,et al.  The development of a scale to measure personal reflection in medical practice and education , 2007, Medical teacher.

[69]  C. M. Dr. Michael J. Weaver MD,et al.  A questionnaire for patients’ evaluations of their physicians’ humanistic behaviors , 2007, Journal of General Internal Medicine.

[70]  G. Miller The assessment of clinical skills/competence/performance , 1990, Academic medicine : journal of the Association of American Medical Colleges.

[71]  R. Hatala,et al.  Assessing the mini‐Clinical Evaluation Exercise in comparison to a national specialty examination , 2006, Medical education.

[72]  S. K. Fields,et al.  Measuring Professionalism: A Review of Studies with Instruments Reported in the Literature between 1982 and 2002 , 2005, Academic medicine : journal of the Association of American Medical Colleges.

[73]  R. Tallis Doctors in society: medical professionalism in a changing world. , 2006, Clinical medicine.

[74]  M. Verkerk,et al.  Reflective professionalism: interpreting CanMEDS’ “professionalism” , 2007, Journal of Medical Ethics.

[75]  R. Kearney Defining professionalism in anaesthesiology , 2005, Medical education.

[76]  G. Eiger,et al.  Do Global Rating Forms Enable Program Directors to Assess the ACGME Competencies? , 2004, Academic medicine : journal of the Association of American Medical Colleges.