A Web-Based Team-Oriented Medical Error Communication Assessment Tool: Development, Preliminary Reliability, Validity, and User Ratings

Background: Multiple-choice exams are not well suited for assessing communication skills. Standardized patient assessments are costly and patient and peer assessments are often biased. Web-based assessment using video content offers the possibility of reliable, valid, and cost-efficient means for measuring complex communication skills, including interprofessional communication. Description: We report development of the Web-based Team-Oriented Medical Error Communication Assessment Tool, which uses videotaped cases for assessing skills in error disclosure and team communication. Steps in development included (a) defining communication behaviors, (b) creating scenarios, (c) developing scripts, (d) filming video with professional actors, and (e) writing assessment questions targeting team communication during planning and error disclosure. Evaluation: Using valid data from 78 participants in the intervention group, coefficient alpha estimates of internal consistency were calculated based on the Likert-scale questions and ranged from α = .79 to α = .89 for each set of 7 Likert-type discussion/planning items and from α = .70 to α = .86 for each set of 8 Likert-type disclosure items. The preliminary test–retest Pearson correlation based on the scores of the intervention group was r = .59 for discussion/planning and r = .25 for error disclosure sections, respectively. Content validity was established through reliance on empirically driven published principles of effective disclosure as well as integration of expert views across all aspects of the development process. In addition, data from 122 medicine and surgical physicians and nurses showed high ratings for video quality (4.3 of 5.0), acting (4.3), and case content (4.5). Conclusions: Web assessment of communication skills appears promising. Physicians and nurses across specialties respond favorably to the tool.