Development and Content Validation of the Information Assessment Method for Patients and Consumers

Background Online consumer health information addresses health problems, self-care, disease prevention, and health care services and is intended for the general public. Using this information, people can improve their knowledge, participation in health decision-making, and health. However, there are no comprehensive instruments to evaluate the value of health information from a consumer perspective. Objective We collaborated with information providers to develop and validate the Information Assessment Method for all (IAM4all) that can be used to collect feedback from information consumers (including patients), and to enable a two-way knowledge translation between information providers and consumers. Methods Content validation steps were followed to develop the IAM4all questionnaire. The first version was based on a theoretical framework from information science, a critical literature review and prior work. Then, 16 laypersons were interviewed on their experience with online health information and specifically their impression of the IAM4all questionnaire. Based on the summaries and interpretations of interviews, questionnaire items were revised, added, and excluded, thus creating the second version of the questionnaire. Subsequently, a panel of 12 information specialists and 8 health researchers participated in an online survey to rate each questionnaire item for relevance, clarity, representativeness, and specificity. The result of this expert panel contributed to the third, current, version of the questionnaire. Results The current version of the IAM4all questionnaire is structured by four levels of outcomes of information seeking/receiving: situational relevance, cognitive impact, information use, and health benefits. Following the interviews and the expert panel survey, 9 questionnaire items were confirmed as relevant, clear, representative, and specific. To improve readability and accessibility for users with a lower level of literacy, 19 items were reworded and all inconsistencies in using a passive or active voice have been solved. One item was removed due to redundancy. The current version of the IAM4all questionnaire contains 28 items. Conclusions We developed and content validated the IAM4all in partnership with information providers, information specialists, researchers and representatives of information consumers. This questionnaire can be integrated within electronic knowledge resources to stimulate users’ reflection (eg, their intention to use information). We claim that any organization (eg, publishers, community organizations, or patient associations), can evaluate and improve their online consumer health information from a consumers’ perspective using this method.

[1]  Faisal Hanif,et al.  The role of quality tools in assessing reliability of the Internet for health information , 2009, Informatics for health & social care.

[2]  Pierre Pluye,et al.  A reflective learning framework to evaluate CME effects on practice reflection , 2010, The Journal of continuing education in the health professions.

[3]  P. Mcgowan,et al.  Web-based education and support for patients on the cardiac surgery waiting list , 2000, Journal of telemedicine and telecare.

[4]  L. Sansom,et al.  Do Internet interventions for consumers cause more harm than good? A systematic review , 2002, Health expectations : an international journal of public participation in health care and health policy.

[5]  Pierre Pluye,et al.  Research Paper: Systematically Assessing the Situational Relevance of Electronic Knowledge Resources: A Mixed Methods Study , 2007, J. Am. Medical Informatics Assoc..

[6]  Michael J Green,et al.  Effect of a computer-based decision aid on knowledge, perceptions, and intentions about genetic testing for breast cancer susceptibility: a randomized controlled trial. , 2004, JAMA.

[7]  D. Roth,et al.  Randomized pilot of a self-guided internet coping group for women with early-stage breast cancer , 2005, Annals of behavioral medicine : a publication of the Society of Behavioral Medicine.

[8]  C. Chronaki,et al.  Informed citizen and empowered citizen in health: results from an European survey , 2011, BMC family practice.

[9]  Robyn Tamblyn,et al.  Assessing the impact of clinical information-retrieval technology in a family practice residency. , 2005, Journal of evaluation in clinical practice.

[10]  P. Pluye,et al.  Using electronic knowledge resources for person-centered medicine – I: An evaluation model , 2011 .

[11]  Pierre Pluye,et al.  Shortcomings of health information on the Internet. , 2003, Health promotion international.

[12]  P. Klemm,et al.  The Internet: friend or foe when providing patient education? , 2008, Clinical journal of oncology nursing.

[13]  D. Finfgeld,et al.  THERAPEUTIC GROUPS ONLINE: THE GOOD, THE BAD, AND THE UNKNOWN , 2000, Issues in mental health nursing.

[14]  S. Haynes,et al.  Content validity in psychological assessment: A functional approach to concepts and methods. , 1995 .

[15]  Roland M. Grad,et al.  Cognitive Impact Assessment of Electronic Knowledge Resources: A Mixed Methods Evaluation Study of a Handheld Prototype , 2006, AMIA.

[16]  Ronald E. Rice,et al.  Influences, usage, and outcomes of Internet health information searching: Multivariate results from the Pew surveys , 2006, Int. J. Medical Informatics.

[17]  M. Schmuckler.  What Is Ecological Validity? A Dimensional Analysis. , 2001, Infancy : the official journal of the International Society on Infant Studies.

[18]  Clarence D Kreiter,et al.  A randomized controlled trial of an information prescription for pediatric patient education on the Internet. , 2004, Archives of pediatrics & adolescent medicine.

[19]  J. Powell,et al.  Primary care Health related virtual communities and electronic support groups : systematic review of the effects of online peer to peer interactions , 2004 .

[20]  Cameron D. Norman,et al.  eHealth Literacy: Essential Skills for Consumer Health in a Networked World , 2006, Journal of medical Internet research.

[21]  Pierre Pluye,et al.  Do Family Physicians Retrieve Synopses of Clinical Research Previously Read as Email Alerts? , 2011, Journal of medical Internet research.

[22]  Pierre Pluye,et al.  Evaluation of email alerts in practice: Part 2. Validation of the information assessment method. , 2010, Journal of evaluation in clinical practice.

[23]  I. Ajzen,et al.  Predicting and Changing Behavior: The Reasoned Action Approach , 2009 .

[24]  Roland M. Grad,et al.  A New Impact Assessment Method to Evaluate Knowledge Resources , 2005, AMIA.

[25]  J. Bernhardt,et al.  Health information-seeking behaviors, health indicators, and health risks. , 2010, American journal of public health.

[26]  Paul B. Kantor,et al.  Studying the Value of Library and Information Services. Part I: Establishing a Theoretical Framework. , 1997 .

[27]  J. Feinglass,et al.  Health Literacy and Mortality Among Elderly Persons , 2007 .

[28]  Roland M. Grad,et al.  Physicians' assessment of the value of clinical information: Operationalization of a theoretical model , 2011, J. Assoc. Inf. Sci. Technol..

[29]  P. Klemm,et al.  Online Cancer Support Groups: A Review of the Research Literature , 2003, Computers, informatics, nursing : CIN.

[30]  Michael J. Miller,et al.  Interactive Digital Education Aid in Breast Reconstruction   , 2008, Plastic and reconstructive surgery.

[31]  Julio Frenk,et al.  Reinventing primary health care: the need for systems integration , 2009, The Lancet.

[32]  W. Whyte,et al.  Participatory Action Research , 1989 .

[33]  Mark Duman,et al.  The state of consumer health information: an overview. , 2009, Health information and libraries journal.

[34]  I. Kickbusch,et al.  Health literacy: addressing the health and education divide. , 2001, Health promotion international.

[35]  Pierre Pluye,et al.  Many family physicians will not manually update PDA software: an observational study. , 2009, Informatics in primary care.

[36]  Heather C. Lench,et al.  Incremental validity of new clinical assessment measures. , 2003, Psychological assessment.

[37]  Bret R. Shaw,et al.  Effects of using online narrative and didactic information on healthcare participation for breast cancer patients. , 2008, Patient education and counseling.

[38]  Alexa T McCray,et al.  Application of Information Technology: Promoting Health Literacy , 2005, J. Am. Medical Informatics Assoc..

[39]  Ha T Tu,et al.  Striking jump in consumers seeking health care information. , 2008, Tracking report.

[40]  Jennie A. Abrahamson,et al.  Lay information mediary behavior uncovered: exploring how nonprofessionals seek health information for themselves and others online. , 2008, Journal of the Medical Library Association : JMLA.

[41]  Alla Keselman,et al.  Developing informatics tools and strategies for consumer-centered health communication. , 2008, Journal of the American Medical Informatics Association : JAMIA.

[42]  Pierre Pluye,et al.  Evaluation of email alerts in practice: Part 1. Review of the literature on clinical emailing channels. , 2010, Journal of evaluation in clinical practice.

[43]  Roland M. Grad,et al.  Four levels of outcomes of information-seeking: A mixed methods study in primary health care , 2013, J. Assoc. Inf. Sci. Technol..

[44]  D. Dwyer,et al.  The impact of consumer health information on the demand for health services , 2013 .

[45]  M. Mettler,et al.  Information therapy: The strategic role of prescribed information in disease self-management. , 2005, Studies in health technology and informatics.

[46]  T. Wagner,et al.  The demand for consumer health information. , 2001, Journal of health economics.

[47]  Cécile R L Boot,et al.  The public and the Internet: Multifaceted drives for seeking health information , 2010, Health Informatics J..

[48]  R. Shegog,et al.  A review of computer and Internet-based interventions for smoking behavior. , 2006, Addictive behaviors.

[49]  C. Dube,et al.  Untangling the Web--the impact of Internet use on health care and the physician-patient relationship. , 2007, Patient education and counseling.

[50]  Pierre Pluye,et al.  Using electronic knowledge resources for person-centered medicine - II: The Number Needed to Benefit from Information (NNBI) , 2011 .

[51]  M. McMullan.  Patients using the Internet to obtain health information: how this affects the patient-health professional relationship. , 2006, Patient education and counseling.

[52]  J Hanson,et al.  Internet usage among women with breast cancer: an exploratory study. , 2000, Clinical breast cancer.

[53]  Heikki Korvenranta,et al.  The effects of Internet or interactive computer-based patient education in the field of breast cancer: a systematic literature review. , 2010, Patient education and counseling.

[54]  Roland M. Grad,et al.  An Online Knowledge Resource and Questionnaires as a Continuing Pharmacy Education Tool to Document Reflective Learning , 2012, American Journal of Pharmaceutical Education.

[55]  Roland M. Grad,et al.  Impact of clinical information-retrieval technology on physicians: A literature review of quantitative, qualitative and mixed methods studies , 2005, Int. J. Medical Informatics.

[56]  Roland M. Grad,et al.  Tensions associated with the use of electronic knowledge resources within clinical decision-making processes: A multiple case study , 2009, Int. J. Medical Informatics.

[57]  P. Pluye,et al.  How information retrieval technology may impact on physician practice: an organizational case study in family medicine. , 2004, Journal of evaluation in clinical practice.

[58]  Andy McEwen,et al.  Online support for smoking cessation: a systematic review of the literature. , 2009, Addiction.

[59]  M. Ebell,et al.  Top 20 research studies of 2011 for primary care physicians. , 2012, American family physician.

[60]  Mark Conner,et al.  Which survey questions change behavior? Randomized controlled trial of mere measurement interventions. , 2010, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[61]  Lynda A. King,et al.  Focus groups in psychological assessment: enhancing content validity by consulting members of the target population. , 2004, Psychological assessment.

[62]  Pierre Pluye,et al.  In pursuit of a valid information assessment method for continuing education: a mixed methods study , 2013, BMC medical education.

[63]  R. J. Cline,et al.  Consumer health information seeking on the Internet: the state of the art. , 2001, Health education research.

[64]  Pierre Pluye,et al.  IAM: A Comprehensive and Systematic Information Assessment Method for Electronic Knowledge Resources , 2009 .

[65]  G. Eysenbach,et al.  The Impact of CyberHealthcare on the Physician–Patient Relationship , 2003, Journal of Medical Systems.