The impact of recommendations and warnings on the quality evaluation of health websites: An online experiment

With the increase in availability of online health information (OHI), consumers need to be able to properly evaluate the quality of health websites. Although several established evaluation criteria for OHI are available, these are rarely used by consumers. To improve people's ability and motivation to critically evaluate OHI, insights into how these criteria can optimally be communicated are needed. This study aims to investigate whether educational messages recommending the use of quality criteria can improve consumers' ability to evaluate OHI credibility, especially among people with low health literacy. We also test whether these messages can yield a stronger effect when combined with information warning consumers against using non-established criteria. In an online experiment, we randomly assigned 403 participants to one of four conditions and asked them to evaluate two websites of different quality. The conditions consisted of recommendations promoting the use of established evaluation criteria, warnings against using commonly adopted non-established criteria, a combination of the two, and a control group. Participants exposed to messages recommending established criteria evaluated the credibility of the lower quality website as poorer, and were better able to discriminate between high and low quality information compared to those in the control group. A combination of recommendations and warnings also improved people's ability to evaluate, but did not yield a stronger effect than the recommendations alone. Subgroup analysis, however, showed that these effects existed only for those with high health literacy. We conclude that providing consumers with criteria to evaluate OHI might be a viable way to improve people's evaluation skills of online health materials. Further research is needed to identify efficient ways to communicate these criteria to low health literate audiences. Strategies to improve consumers' ability to evaluate online information are needed.Recommendations about established criteria positively affect credibility evaluation.Adding warnings against non-established criteria does not improve evaluation.Recommendations only work for high health literate consumers.Future research should find viable ways to reach low health literate consumers.