Does question order influence sensitivity to scope? Empirical findings from a web-based contingent valuation study

This paper examines the relationship between question order and sensitivity to scope in a large-scale web-based stated preference survey using an Internet panel. Results are presented from a contingent valuation survey in which each individual was asked to value two independent gains in life expectancy. Using split-sample data for question order we are able to study sensitivity to scope by carrying out an internal as well as an external scope test, and to examine whether question order influences sensitivity to scope. Finally, we address whether our results raise some specific concerns regarding the use of web-based surveys. Overall we find that choice of elicitation approach – in this case bottom-up versus top-down – influences the stated willingness-to-pay values leading to order effects and differences in scope sensitivity. Our findings demonstrate that elicitation approach affects the decision-making strategy, suggesting that preferences at least to some extent are reference dependent and constructed during the elicitation task. In addition, our findings indicate some relation between scope insensitivity, time spent on filling out the questionnaire and experience as an Internet panel member.