Evaluating consumer health information websites: The importance of collecting observational, user-driven data

The internet is an increasingly popular source of consumer health information (CHI). Worldwide, millions of people regularly visit CHI websites to seek answers to medical issues, either for themselves or their loved ones. As CHI websites play such a vital role in people's health behaviour, the evaluation of these sites has often been addressed in scholarly studies. The vast majority of these studies have focused on evaluation methods for assessing the quality of CHI websites (expert testing). While this emphasis on quality is understandable — health information needs to be accurate and up-to-date — calls are increasingly made to not only ensure the quality of information on CHI websites, but to also focus on the usability of these sites. This paper offers a review of the usability studies that have so far been conducted regarding CHI websites. We will discuss the potential of, in particular, observational user-driven research for CHI websites, by discussing the methodological drawbacks of the existing usability studies. In addition, we will examine the role of user characteristics in the evaluation of CHI web sites. The results of our review indicate that observational user research, when conducted properly, may offer valuable insights that complement those gained by means of expert testing.

[1]  M. J. van den Haak,et al.  A penny for your thoughts. Investigating the validity and reliability of think-aloud protocols for usability testing , 2008 .

[2]  G. Kok,et al.  Testing the usability of access structures in a health education brochure. , 2007, British journal of health psychology.

[3]  Rowena Cullen,et al.  Democracy online: an assessment of New Zealand government web sites , 2000, Gov. Inf. Q..

[4]  Valerie Njie,et al.  Internet Usage by Low-Literacy Adults Seeking Health Information: An Observational Analysis , 2004, Journal of medical Internet research.

[5]  Monique W. M. Jaspers,et al.  A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence , 2009, Int. J. Medical Informatics.

[6]  Gbogboade Ademiluyi,et al.  Evaluating the reliability and validity of three tools to assess the quality of health information on the Internet. , 2003, Patient education and counseling.

[7]  Elizabeth Sillence,et al.  A framework for understanding trust factors in web-based health advice , 2006, Int. J. Hum. Comput. Stud..

[8]  Christian Köhler,et al.  How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews , 2002, BMJ : British Medical Journal.

[9]  Gunther Eysenbach,et al.  Design and Evaluation of Consumer Health Information Web Sites , 2005 .

[10]  Joseph H. Goldberg,et al.  Eye tracking in web search tasks: design implications , 2002, ETRA.

[11]  Elmer V. Bernstam,et al.  Instruments to assess the quality of health information on the World Wide Web: what can our patients actually use? , 2005, Int. J. Medical Informatics.

[12]  E. Seydel,et al.  Evaluation of Internet-Based Technology for Supporting Self-Care: Problems Encountered by Patients and Caregivers When Using Self-Care Applications , 2008, Journal of medical Internet research.

[13]  Ronald E. Rice,et al.  Influences, usage, and outcomes of Internet health information searching: Multivariate results from the Pew surveys , 2006, Int. J. Medical Informatics.

[14]  R. J. Cline,et al.  Consumer health information seeking on the Internet: the state of the art. , 2001, Health education research.

[15]  Margarette E. Beckwith,et al.  The Role of User Input in Shaping Online Information From the National Cancer Institute , 2005, Journal of medical Internet research.

[16]  Sue McKemmish,et al.  Consumer empowerment through metadata-based information quality reporting: The Breast Cancer Knowledge Online Portal , 2008 .

[17]  Lee Priest,et al.  Website task performance by older adults , 2007, Behav. Inf. Technol..

[18]  Joseph Sharit,et al.  Usability of the Medicare health web site. , 2008, JAMA.

[19]  Matthijs Sienot.  Pretesting Web Sites , 1997 .

[20]  Raquel Benbunan-Fich,et al.  Using protocol analysis to evaluate the usability of a commercial web site , 2001, Inf. Manag..

[21]  Menno D. T. de Jong,et al.  Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue , 2003, Behav. Inf. Technol..

[22]  Paul Kim,et al.  Published criteria for evaluating health related web sites: review , 1999, BMJ.

[23]  Alexa T. McCray,et al.  Usability issues in developing a Web-based consumer health site , 2000, AMIA.

[24]  Alla Keselman,et al.  Exploring User Navigation during Online Health Information Seeking , 2006, AMIA.

[25]  Robert A. Greenes,et al.  Positive attitudes and failed queries: an exploration of the conundrums of consumer health information retrieval , 2004, Int. J. Medical Informatics.

[26]  Peter R. Harris,et al.  How do patients evaluate and make use of online health information? , 2007, Social science & medicine.

[27]  Menno D. T. de Jong,et al.  Evaluating municipal websites: A methodological comparison of three think-aloud variants , 2009, Gov. Inf. Q..

[28]  Nicole Campbell,et al.  Usability assessment of library-related web sites : methods and case studies , 2001 .

[29]  James G. Anderson.  Consumers of e-Health , 2004 .

[30]  Derek L. Hansen,et al.  Adolescents Searching for Health Information on the Internet: An Observational Study , 2003, Journal of medical Internet research.

[31]  L. Gilbert,et al.  Evaluating a teen STD prevention Web site. , 2005, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[32]  Katherine A Prentice,et al.  Web usability testing with a Hispanic medically underserved population. , 2009, Journal of the Medical Library Association : JMLA.

[33]  C. Chronaki,et al.  European citizens' use of E-health services: A study of seven countries , 2007, BMC public health.

[34]  Peter Williams,et al.  Surfing for health: user evaluation of a health information website. Part two: fieldwork. , 2002, Health information and libraries journal.

[35]  Jane Greenberg,et al.  Relevance criteria identified by health information users during Web searches , 2006, J. Assoc. Inf. Sci. Technol..

[36]  W Kerry Mummery,et al.  Applying usability testing techniques to improve a health promotion website. , 2008, Health promotion journal of Australia : official journal of Australian Association of Health Promotion Professionals.

[37]  Brenda L. Battleson,et al.  Usability testing of an academic library Web site: a case study , 2001 .

[38]  Monique W. M. Jaspers,et al.  Usability Studies on Interactive Health Information Systems; Where Do We Stand? , 2008, MIE.

[39]  Huong Q. Nguyen,et al.  Designing and Testing a Web-Based Interface for Self-monitoring of Exercise and Symptoms for Older Adults With Chronic Obstructive Pulmonary Disease , 2009, Computers, informatics, nursing : CIN.

[40]  J. Powell,et al.  Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. , 2002, JAMA.

[41]  Gilan El Saadawi,et al.  Caring Connection: Developing an Internet Resource for Family Caregivers of Children with Cancer , 2005, AMIA.

[42]  Alla Keselman,et al.  Research Paper: Consumer Health Information Seeking as Hypothesis Testing , 2008, J. Am. Medical Informatics Assoc..

[43]  Bryant Thomas Karras,et al.  ActiveOptions: Leveraging existing knowledge and usability testing to develop a physical activity program website for older adults , 2007, AMIA.

[44]  Peter A. Bath,et al.  Assessing the quality of websites providing information on multiple sclerosis: evaluating tools and comparing sites , 2007, Health Informatics J..