Health Information Seals of Approval: What do they Signify?

Much of the health information available to consumers on the Internet is incomplete, out of date and even inaccurate. Seals of approval or trustmarks have been suggested as a strategy to assist consumers in identifying high-quality information. Little is known, however, about how consumers interpret such seals. This study addresses this issue by examining assumptions about the quality criteria that are reflected by a seal of approval. This question is of particular importance because a wide variety of quality criteria have been suggested for online health information, including: core aspects of quality such as accuracy, currency and completeness; proxy indicators of quality such as the disclosure of commercial interests; and indicators that reflect the quality of the site or the interaction it affords, such as the availability of a search mechanism. The results of this study suggest that seals of approval are assumed to certify information quality primarily with respect to core quality indicators, aspects that subjects both consider to be important and feel relatively less able to evaluate for themselves (compared with their ability to rate proxy indicators of information or indicators of site or interaction quality). This assumption is largely inconsistent with practice: most seals of approval involve assessment of proxy indicators of information quality, rather than direct assessment of content. These results identify a problem that certification or accreditation bodies must address since, unless and until consumer expectations are congruent with evaluation practice, seals of approval will seem to promise more than they deliver.

[1]  Petra Wilson,et al.  The quality of health information on the internet , 2002, BMJ : British Medical Journal.

[2]  T. L. Parkinson The Role of Seals and Certifications of Approval in Consumer Decision‐Making. , 1975 .

[3]  J. Burkell,et al.  The dilemma of survey nonresponse , 2003 .

[4]  T. L. Lissman,et al.  A critical review of internet information about depression. , 2001, Psychiatric services.

[5]  Sam Karp,et al.  Quality of healthcare information on the Internet: caveat emptor still rules. , 2002, Managed care quarterly.

[6]  Maurizio Bonati,et al.  Follow up of quality of public oriented health information on the world wide web: systematic re-evaluation , 2002, BMJ : British Medical Journal.

[7]  G D Lundberg,et al.  Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. , 1997, JAMA.

[8]  S. Singletary,et al.  Breast cancer on the world wide web: cross sectional survey of quality of information and popularity of websites , 2002, BMJ : British Medical Journal.

[9]  Michael Kostrzewa,et al.  Quality of drug information on the World Wide Web and strategies to improve pages with poor information quality. An intervention study on pages about sildenafil. , 2003, British journal of clinical pharmacology.

[10]  Anthony D. Miyazaki,et al.  Internet Seals of Approval: Effects on Online Privacy Policies and Consumer Perceptions , 2002 .

[11]  Ron D. Appel,et al.  The Health On the Net Code of Conduct for medical and health Web sites , 1998, MedInfo.

[12]  Alastair G. Smith,et al.  Testing the Surf: Criteria for Evaluating Internet Information Resources , 1997 .

[13]  Boyd R. Collins Beyond Cruising: Reviewing. , 1996 .

[14]  H. Christensen,et al.  Quality of web based information on treatment of depression: cross sectional survey , 2000, BMJ : British Medical Journal.

[15]  Walter Minkel No, It's Not All True. , 2000 .

[16]  S. Joy Mountford,et al.  Tools and techniques for creative design , 1995 .

[17]  Paul Kim,et al.  Published criteria for evaluating health related web sites: review , 1999, BMJ.

[18]  Heidelberg Consensus Recommendations on Trustmarks , 2000, Journal of Medical Internet Research.

[19]  Manish Latthe,et al.  Accuracy of information on apparently credible websites: survey of five common health topics , 2002, BMJ : British Medical Journal.

[20]  D. T. Hawkins What is credible information , 1999 .

[21]  Gunther Eysenbach,et al.  Consumer health informatics , 2000, BMJ : British Medical Journal.

[22]  Tschera Harkness Connell,et al.  Testing the Accuracy of Information on the World Wide Web Using the AltaVista Search Engine , 1999 .

[23]  Carol L. Barry,et al.  Users' Criteria for Relevance Evaluation: A Cross-situational Comparison , 1998, Inf. Process. Manag..

[24]  Robert Kiley Consumer health information on the Internet , 1998, Journal of the Royal Society of Medicine.

[25]  M. Johnston The quality of health , 1995 .

[26]  G. Eysenbach Infodemiology: The epidemiology of (mis)information. , 2002, The American journal of medicine.

[27]  V P Abbott,et al.  Web page quality: can we measure it and what do we find? A report of exploratory findings. , 2000, Journal of public health medicine.

[28]  B. J. Fogg,et al.  Elements that affect web credibility: early results from a self-report study , 2000, CHI Extended Abstracts.

[29]  Miriam J. Metzger,et al.  Internet use in the contemporary media environment. , 2001 .

[30]  T Delamothe,et al.  Quality of websites: kitemarking the west wind , 2000, BMJ : British Medical Journal.

[31]  Atelier van Lieshout The good, the bad + the ugly , 1998 .

[32]  Mark A. Musen,et al.  The low availability of metadata elements for evaluating the quality of medical information on the World Wide Web , 1999, AMIA.

[33]  R. Smith Almost no evidence exists that the internet harms health. , 2001, BMJ.

[34]  Peter Williams,et al.  Digital health information provision and health outcomes , 2001, J. Inf. Sci..

[35]  P. Impicciatore,et al.  Reliability of health information for the public on the world wide web: systematic survey of advice on managing fever in children at home , 1997, BMJ.

[36]  Jacquelyn A. Burkell,et al.  Believe it or not: Factors influencing credibility on the Web , 2002, J. Assoc. Inf. Sci. Technol..

[37]  A. Jadad,et al.  Analysis of cases of harm associated with use of health information on the internet. , 2002, JAMA.

[38]  Christian Köhler,et al.  How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews , 2002, BMJ : British Medical Journal.

[39]  D. Sarel,et al.  Consumer (Mis)Perceptions and usage of Third Party Certification Marks, 1972 and 1980: Did Public Policy have an Impact? , 1981 .

[40]  P Impicciatore,et al.  Parents on the Web: Risks for Quality Management of Cough in Children , 2000, Pediatrics.

[41]  Ahmad Risk,et al.  Review Of Internet Health Information Quality Initiatives , 2001, Journal of medical Internet research.

[42]  Edwin R. Stafford,et al.  Comprehension and Perceived Believability of Seals of Approval Information in Advertising , 1993 .

[43]  Jonathan Marshall,et al.  The impact of displayed awards on the credibility and retention of Web site information , 2000, AMIA.

[44]  Nancy Everhart,et al.  Web Page Evaluation Worksheet. , 1998 .

[45]  Dan Brickley,et al.  A Framework for Improving the Quality of Health Information on the World-Wide-Web and Bettering Public (E-)Health: The Medcertain Approach , 2001, MedInfo.

[46]  Samer Faraj,et al.  The Role of Intermediaries in the Development of Trust on the WWW: The Use and Prominence of Trusted Third Parties and Privacy Statements , 2006, J. Comput. Mediat. Commun..

[47]  Lee Rainie,et al.  The online health care revolution: how the web helps americans take better care of themselves , 2000 .

[48]  J. Powell,et al.  Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. , 2002, JAMA.

[49]  M. Latthe,et al.  Quality of medical information about menorrhagia on the worldwide web , 2000, BJOG : an international journal of obstetrics and gynaecology.

[50]  G Eysenbach,et al.  Information in practice Towards quality management of medical information on the internet : evaluation , labelling , and filtering of information , 1998 .

[51]  Jeremy C Wyatt,et al.  Survey of Doctors' Experience of Patients Using the Internet , 2002, Journal of medical Internet research.

[52]  Christian Köhler,et al.  Does the internet harm health? Database of adverse events related to the internet has been set up. , 2002, BMJ.

[53]  J Sybil Biermann,et al.  Melanoma information on the Internet: often incomplete--a public health opportunity? , 2002, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[54]  M. Winker,et al.  Guidelines for medical and health information sites on the internet: principles governing AMA web sites. American Medical Association. , 2000, JAMA.

[55]  H. Sandvik Health information and interaction on the internet: a survey of female urinary incontinence , 1999, BMJ.

[56]  A R Jadad,et al.  Rating health information on the Internet: navigating to knowledge or to Babel? , 1998, JAMA.

[57]  Kylie A. Williams,et al.  How do Consumers Search for and Appraise Information on Medicines on the Internet? A Qualitative Study Using Focus Groups , 2003, Journal of medical Internet research.

[58]  Don Fallis,et al.  Research Paper: Indicators of Accuracy of Consumer Health Information on the Internet: A Study of Indicators Relating to Information for Managing Fever in Children in the Home , 2002, J. Am. Medical Informatics Assoc..