The Use of Web-Based Patient Reviews to Assess Medical Oncologists’ Competency: Mixed Methods Sequential Explanatory Study

Background Patients increasingly use web-based evaluation tools to assess their physicians, health care teams, and overall medical experience. Objective This study aimed to evaluate the extent to which the standardized physician competencies of the CanMEDS Framework are present in web-based patient reviews (WPRs) and to identify patients’ perception of important physician qualities in the context of quality cancer care. Methods The WPRs of all university-affiliated medical oncologists in midsized cities with medical schools in the province of Ontario (Canada) were collected. Two reviewers (1 communication studies researcher and 1 health care professional) independently assessed the WPRs according to the CanMEDS Framework and identified common themes. Comment scores were then evaluated to identify κ agreement rates between the reviewers, and a descriptive quantitative analysis of the cohort was completed. Following the quantitative analysis, an inductive thematic analysis was performed. Results This study identified 49 actively practicing university-affiliated medical oncologists in midsized urban areas in Ontario. A total of 473 WPRs reviewing these 49 physicians were identified. Among the CanMEDS competencies, those defining the roles of medical experts, communicators, and professionals were the most prevalent (303/473, 64%; 182/473, 38%; and 129/473, 27%, respectively). Common themes in WPRs include medical skill and knowledge, interpersonal skills, and answering questions (from the patient to the physician). Detailed WPRs tend to include the following elements: experience and connection; discussion and evaluation of the physician’s knowledge, professionalism, interpersonal skills, and punctuality; in positive reviews, the expression of feelings of gratitude and a recommendation; and in negative reviews, discouragement from seeking the physician’s care. Patients’ perception of medical skills is less specific than their perception of interpersonal qualities, although medical skills are the most commented-on element of care in WPRs. Patients’ perception of interpersonal skills (listening, compassion, and overall caring demeanor) and other experiential phenomena, such as feeling rushed during appointments, is often specific and detailed. Details about a physician’s interpersonal skills or “bedside manner” are highly perceived, valued, and shareable in an WPR context. A small number of WPRs reflected a distinction between the value of medical skills and that of interpersonal skills. The authors of these WPRs claimed that for them, a physician’s medical skills and competence are more important than their interpersonal skills. Conclusions CanMEDS roles and competencies that are explicitly patient facing (ie, those directly experienced by patients in their interactions with physicians and through the care that physicians provide) are the most likely to be present and reported on in WPRs. The findings demonstrate the opportunity to learn from WPRs, not simply to discern physicians’ popularity but to grasp what patients may expect from their physicians. In this context, WPRs can represent a method for the measurement and assessment of patient-facing physician competency.

[1]  Alexander J. Schupper,et al.  Trends in Online Patient Perspectives of Neurosurgeons: A Sentiment Analysis , 2022, Neurosurgery Open.

[2]  J. Pham,et al.  Online Patient Reviews of Breast Reconstruction: RealSelf Analysis , 2022, Plastic and reconstructive surgery. Global open.

[3]  Sonja Bidmon,et al.  The Impact of Social Influence on the Intention to Use Physician Rating Websites: Moderated Mediation Analysis Using a Mixed Methods Approach , 2022, Journal of medical Internet research.

[4]  G. Gao,et al.  Recency of Online Physician Ratings. , 2022, JAMA internal medicine.

[5]  Elissa V. Klinger,et al.  Patient Experience and Satisfaction in Online Reviews of Obstetric Care: Observational Study , 2022, JMIR formative research.

[6]  Panhao Huang,et al.  The Effect of Online Health Information Seeking on Physician-Patient Relationships: Systematic Review , 2020, Journal of medical Internet research.

[7]  S. Chou,et al.  Comparing the Impact of Online Ratings and Report Cards on Patient Choice of Cardiac Surgeon: Large Observational Study. , 2021, Journal of medical Internet research.

[8]  David A. Broniatowski,et al.  Questioning the Yelp Effect: Mixed Methods Analysis of Web-Based Reviews of Urgent Cares , 2021, Journal of medical Internet research.

[9]  S. McLennan,et al.  One Decade of Online Patient Feedback: Longitudinal Analysis of Data From a German Physician Rating Website , 2021, Journal of medical Internet research.

[10]  A. Guddati,et al.  Cyberspace and Libel: A Dangerous Balance for Physicians , 2021, Interactive journal of medical research.

[11]  Cheryl Rathert,et al.  Are Online Patient Reviews Associated With Health Care Outcomes? A Systematic Review of the Literature , 2021, Medical care research and review : MCRR.

[12]  Yiwei Chen,et al.  User-Generated Physician Ratings and Their Effects on Patients' Physician Choices: Evidence from Yelp , 2021, SSRN Electronic Journal.

[13]  Yang Wang,et al.  The Influence of Doctors’ Online Reputation on the Sharing of Outpatient Experiences: Empirical Study , 2020, Journal of medical Internet research.

[14]  Yongmei Liu,et al.  The Mediating Role of Patients’ Trust Between Web-Based Health Information Seeking and Patients’ Uncertainty in China: Cross-sectional Web-Based Survey , 2020, Journal of medical Internet research.

[15]  T. Daskivich,et al.  Correlation of Online Physician Rating Subscores and Association With Overall Satisfaction: Observational Study of 212,933 Providers , 2020, Journal of medical Internet research.

[16]  M. Kanter,et al.  A Comparison of Online Physician Ratings and Internal Patient-Submitted Ratings from a Large Healthcare System , 2019, Journal of General Internal Medicine.

[17]  D. Studdert,et al.  Identification of practitioners at high risk of complaints to health profession regulators , 2019, BMC Health Services Research.

[18]  B. Breyer,et al.  Online physician reviews: is there a place for them? , 2019, Risk management and healthcare policy.

[19]  Richard L Street,et al.  What Do Patients Say About Doctors Online? A Systematic Review of Studies on Patient Online Reviews , 2018, Journal of medical Internet research.

[20]  P. Culligan,et al.  Understanding Your Online Ratings: A Methodological Analysis Using Urogynecologists in the United States , 2019, Female pelvic medicine & reconstructive surgery.

[21]  P. Beredjiklian,et al.  Physician Rating Websites: an Analysis of Physician Evaluation and Physician Perception. , 2019, The archives of bone and joint surgery.

[22]  Y. Nasseri,et al.  “But His Yelp Reviews Are Awful!”: Analysis of General Surgeons’ Yelp Reviews , 2018, Journal of medical Internet research.

[23]  David M. Ray,et al.  Online physician review websites poorly correlate to a validated metric of patient satisfaction. , 2018, The Journal of surgical research.

[24]  Robert J. McGrath,et al.  The Validity of Online Patient Ratings of Physicians: Analysis of Physician Peer Reviews and Patient Ratings , 2018, Interactive journal of medical research.

[25]  Garth Fuller,et al.  Online physician ratings fail to predict actual performance on measures of quality, value, and peer review , 2018, J. Am. Medical Informatics Assoc..

[26]  Pooja S. Parameshwar,et al.  Online Physician Reviews in Female Pelvic Medicine and Reconstructive Surgery: What Do Patients Really Want? , 2018, Female pelvic medicine & reconstructive surgery.

[27]  Deborah E. White,et al.  Thematic Analysis , 2017 .

[28]  A. Daluiski,et al.  Online Patient Ratings: Why They Matter and What They Mean. , 2016, The Journal of hand surgery.

[29]  Mark Dredze,et al.  A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews , 2014, J. Am. Medical Informatics Assoc..

[30]  Ralf Terlutter,et al.  Who Uses Physician-Rating Websites? Differences in Sociodemographic Variables, Psychographic Variables, and Health Status of Users and Nonusers of Physician-Rating Websites , 2014, Journal of medical Internet research.

[31]  M. Elliott,et al.  Specialties Differ in Which Aspects of Doctor Communication Predict Overall Physician Ratings , 2014, Journal of General Internal Medicine.

[32]  U. Flick,et al.  The SAGE Handbook of Qualitative Data Analysis , 2013 .

[33]  Uwe Sander,et al.  Physician Choice Making and Characteristics Associated With Using Physician-Rating Websites: Cross-Sectional Study , 2013, Journal of medical Internet research.

[34]  Michael J Sacopulos,et al.  Online Doctor Reviews: Do They Track Surgeon Volume, a Proxy for Quality of Care? , 2012, Journal of medical Internet research.

[35]  A. Jha,et al.  A Changing Landscape of Physician Quality Reporting: Analysis of Patients’ Online Ratings of Their Physicians Over a 5-Year Period , 2012, Journal of medical Internet research.

[36]  U. Sarkar,et al.  What Patients Say About Their Doctors Online: A Qualitative Content Analysis , 2012, Journal of General Internal Medicine.

[37]  Alex Macario,et al.  Analysis of 4999 Online Physician Ratings Indicates That Most Patients Give Physicians a Favorable Rating , 2011, Journal of medical Internet research.

[38]  B. Mavis,et al.  Female Patients’ Preferences Related to Interpersonal Communications, Clinical Competence, and Gender When Selecting a Physician , 2005, Academic medicine : journal of the Association of American Medical Colleges.

[39]  R. Oliver A Cognitive Model of the Antecedents and Consequences of Satisfaction Decisions , 1980 .

[40]  Kangyoon Lee,et al.  The Role of Emotions Intensity in Helpfulness of Online Physician Reviews , 2022, Intelligent Automation & Soft Computing.

[41]  A. K. A. M. Mph,et al.  “Told”: the Word Most Correlated to Negative Online Hospital Reviews , 2019, Journal of General Internal Medicine.

[42]  R. Barbour Quality of Data Analysis , 2014 .

[43]  A. Panter,et al.  APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. , 2012 .

[44]  J. C. Meakins,et al.  The Royal College of Physicians and Surgeons of Canada , 1936, Canadian Medical Association journal.