Quality of health information for consumers on the web: A systematic review of indicators, criteria, tools, and evaluation results

The quality of online health information for consumers has been a critical issue that concerns all stakeholders in healthcare. To gain an understanding of how quality is evaluated, this systematic review examined 165 articles in which researchers evaluated the quality of consumer‐oriented health information on the web against predefined criteria. It was found that studies typically evaluated quality in relation to the substance and formality of content, as well as to the design of technological platforms. Attention to design, particularly interactivity, privacy, and social and cultural appropriateness is on the rise, which suggests the permeation of a user‐centered perspective into the evaluation of health information systems, and a growing recognition of the need to study these systems from a social‐technical perspective. Researchers used many preexisting instruments to facilitate evaluation of the formality of content; however, only a few were used in multiple studies, and their validity was questioned. The quality of content (i.e., accuracy and completeness) was always evaluated using proprietary instruments constructed based on medical guidelines or textbooks. The evaluation results revealed that the quality of health information varied across medical domains and across websites, and that the overall quality remained problematic. Future research is needed to examine the quality of user‐generated content and to explore opportunities offered by emerging new media that can facilitate the consumer evaluation of health information.

[1]  H. Sandvik Health information and interaction on the internet: a survey of female urinary incontinence , 1999, BMJ.

[2]  Petra Wilson,et al.  How to find the good and avoid the bad or ugly: a short guide to tools for rating quality of health information on the internet. , 2002, BMJ : British Medical Journal.

[3]  A. Jadad,et al.  Analysis of cases of harm associated with use of health information on the internet. , 2002, JAMA.

[4]  Peter A. Bath,et al.  Assessing the quality of websites providing information on multiple sclerosis: evaluating tools and comparing sites , 2007, Health Informatics J..

[5]  Yunli Wang,et al.  Automatic detecting indicators for quality of health information on the Web , 2007, Int. J. Medical Informatics.

[6]  Barak Gaster,et al.  A Critical Assessment of Diabetes Information on the Internet , 2002 .

[7]  Ben Shneiderman,et al.  eHealth research from the user's perspective. , 2007, American journal of preventive medicine.

[8]  Alejandro R Jadad,et al.  Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination , 2002, BMJ : British Medical Journal.

[9]  Yasser Khazaal,et al.  Quality of web-based information on alcohol dependence , 2009 .

[10]  R. Ní Ríordáin,et al.  Head and neck cancer information on the internet: type, accuracy and content. , 2009, Oral oncology.

[11]  Leslie S. Zun,et al.  Completeness and Accuracy of Emergency Medical Information on the Web: Update 2008 , 2011, The western journal of emergency medicine.

[12]  Markus A. Feufel,et al.  What do Web-Use Skill Differences Imply for Online Health Information Searches? , 2012, Journal of medical Internet research.

[13]  B. J. Fogg,et al.  Credibility and computing technology , 1999, CACM.

[14]  D Charnock,et al.  DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. , 1999, Journal of epidemiology and community health.

[15]  B. Tan,et al.  A Review of Mesothelioma Information on the World Wide Web , 2009, Journal of thoracic oncology : official publication of the International Association for the Study of Lung Cancer.

[16]  S. Singletary,et al.  Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites , 2002, BMJ : British Medical Journal.

[17]  Kevin O. Hwang,et al.  Quality of weight loss advice on internet forums. , 2007, The American journal of medicine.

[18]  Debra L Roter,et al.  Survey of quality, readability, and social reach of websites on osteosarcoma in adolescents. , 2013, Patient education and counseling.

[19]  M Tallgren,et al.  Patient information about general anaesthesia on the internet , 2009, Anaesthesia.

[20]  R. Croyle,et al.  Frustrated and Confused: The American Public Rates its Cancer-Related Information-Seeking Experiences , 2008, Journal of General Internal Medicine.

[21]  W. Chou,et al.  Web 2.0 for health promotion: reviewing the current evidence. , 2013, American journal of public health.

[22]  G. Eysenbach Medicine 2.0: Social Networking, Collaboration, Participation, Apomediation, and Openness , 2008, Journal of medical Internet research.

[23]  V P Abbott,et al.  Web page quality: can we measure it and what do we find? A report of exploratory findings. , 2000, Journal of public health medicine.

[24]  Heiko Spallek,et al.  English and Spanish oral cancer information on the internet: a pilot surface quality and content evaluation of oral cancer web sites. , 2011, Journal of public health dentistry.

[25]  Lisa Hartling,et al.  Social media use among patients and caregivers: a scoping review , 2013, BMJ Open.

[26]  Beata Bajorek,et al.  Assessing the quality, suitability and readability of internet-based health information about warfarin for patients. , 2012, The Australasian medical journal.

[27]  E. Brunswik,et al.  The Conceptual Framework of Psychology , 1954 .

[28]  James B. Williams,et al.  Social networking applications in health care: threats to the privacy and security of health information , 2010, SEHC '10.

[29]  Arch G. Mainous,et al.  The accuracy of nutrition information on the Internet for type 2 diabetes. , 2010, Archives of internal medicine.

[30]  J. Stinson,et al.  Perspectives on quality and content of information on the internet for adolescents with cancer , 2011, Pediatric blood & cancer.

[31]  I G Chestnutt The nature and quality of periodontal related patient information on the world-wide web , 2002, British Dental Journal.

[32]  M. Hamady,et al.  Analysis of the Quality of Information Obtained About Uterine Artery Embolization From the Internet , 2012, CardioVascular and Interventional Radiology.

[33]  G Eysenbach,et al.  Information in practice Towards quality management of medical information on the internet : evaluation , labelling , and filtering of information , 1998 .

[34]  Amy S. Hedman,et al.  Using the SMOG Formula to Revise a Health-Related Document , 2008 .

[35]  Sriharsha Athreya,et al.  Assessing the Reliability and Quality of Online Uterine Fibroid Embolization Resources , 2013, CardioVascular and Interventional Radiology.

[36]  B. J. Fogg,et al.  Persuasive technology: using computers to change what we think and do , 2002, UBIQ.

[37]  Steven Muncer,et al.  Reliability of Health Information on the Internet: An Examination of Experts' Ratings , 2002, Journal of medical Internet research.

[38]  Elmer V. Bernstam,et al.  Instruments to assess the quality of health information on the World Wide Web: what can our patients actually use? , 2005, Int. J. Medical Informatics.

[39]  Osman H Ahmed,et al.  Acute low back pain information online: an evaluation of quality, content accuracy and readability of related websites. , 2012, Manual therapy.

[40]  M. Bailey,et al.  Quality and readability of online patient information for abdominal aortic aneurysms. , 2012, Journal of vascular surgery.

[41]  Yasser Khazaal,et al.  Quality of web‐based information on depression , 2010, Depression and anxiety.

[42]  Elmer V. Bernstam,et al.  Usability of quality measures for online health information: Can commonly used technical quality criteria be reliably assessed? , 2005, Int. J. Medical Informatics.

[43]  Josip Car,et al.  Interventions for enhancing consumers' online health literacy. , 2011, The Cochrane database of systematic reviews.

[44]  J. Powell,et al.  Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. , 2002, JAMA.

[45]  Julie Ann Sosa,et al.  Filling a Void: Thyroid Cancer Surgery Information on the Internet , 2007, World Journal of Surgery.

[46]  G D Lundberg,et al.  Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. , 1997, JAMA.

[47]  F. Pérez-López,et al.  Assessing the content and quality of information on the treatment of postmenopausal osteoporosis on the World Wide Web , 2006, Gynecological endocrinology : the official journal of the International Society of Gynecological Endocrinology.

[48]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[49]  B. Fogg The ethics of persuasive technology , 2003 .

[50]  Katrin Baumgartner,et al.  Persuasion Theory And Research , 2016 .

[51]  David Hawking,et al.  Automated Assessment of the Quality of Depression Websites , 2005, Journal of medical Internet research.

[52]  R. J. Cline,et al.  Consumer health information seeking on the Internet: the state of the art. , 2001, Health education research.

[53]  Ahmad Risk,et al.  Review Of Internet Health Information Quality Initiatives , 2001, Journal of medical Internet research.

[54]  J C Wyatt,et al.  Commentary: measuring quality and impact of the world wide web , 1997, BMJ.

[55]  Aris Antsaklis,et al.  Polycystic ovary syndrome: double click and right check. What do patients learn from the Internet about PCOS? , 2012, European journal of obstetrics, gynecology, and reproductive biology.

[56]  D. O’Keefe Persuasion , 1990, The Handbook of Communication Skills.

[57]  Yan Zhang,et al.  Beyond quality and accessibility: Source selection in consumer health information searching , 2014, J. Assoc. Inf. Sci. Technol..

[58]  Laura O'Grady,et al.  Future directions for depicting credibility in health care web sites , 2006, Int. J. Medical Informatics.

[59]  Lori W Turner,et al.  Evaluation of web-based osteoporosis educational materials. , 2005, Journal of women's health.

[60]  Philip B. Mitchell,et al.  Review of the Quality of Information on Bipolar Disorder on the Internet , 2009 .

[61]  Christian Köhler,et al.  How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews , 2002, BMJ : British Medical Journal.

[62]  Paul Kim,et al.  Published criteria for evaluating health related web sites: review , 1999, BMJ.

[63]  Yan Zhang,et al.  Consumer health information searching process in real life settings , 2012, ASIST.

[64]  A R Jadad,et al.  Rating health information on the Internet: navigating to knowledge or to Babel? , 1998, JAMA.