Learning in practice Review of instruments for peer assessment of physicians

Abstract Objectives To identify existing instruments for rating peers (professional colleagues) in medical practice and to evaluate them in terms of how they have been developed, their validity and reliability, and their appropriateness for use in clinical settings, including primary care. Design Systematic literature review. Data sources Electronic search techniques, snowball sampling, and correspondence with specialists. Study selection The peer assessment instruments identified were evaluated in terms of how they were developed and to what extent, if relevant, their psychometric properties had been determined. Results A search of six electronic databases identified 4566 possible articles. After appraisal of the abstracts and in depth assessment of 42 articles, three rating scales fulfilled the inclusion criteria and were fully appraised. The three instruments did not meet established standards of instrument development, as no reference was made to a theoretical framework and the published psychometric data omitted essential work on construct and criterion validity. Rater training was absent, and guidance consisted of short written instructions. Two instruments were developed for a hospital setting in the United States and one for a primary care setting in Canada. Conclusions The instruments developed to date for physicians to evaluate characteristics of colleagues need further assessment of validity before their widespread use is merited.