Best-worst scaling: consistency of preferences with discrete choice experiments and stability over time

This paper addresses two research questions regarding best-worst scaling (BWS): (a) Are preferences estimated from BWS tasks consistent with those from discrete choice experiments (DCEs)? and (b)  Are population preferences estimated from BWS tasks stable over time? The context of this research was the estimation of a set of preference weights for a number of social care related quality of life indicators, which are used to describe dimensions of people's lives that may be expected to be influenced by social care. This research programme was funded by the UK Department of Health through their HTA programme, and the UK Office for National Statistics through the Treasury's ‘Invest to Save' programme. To address the first question, 300 interviews were undertaken with members of the general public in which they were asked to complete both DCEs and BWS tasks. This approach allowed efficient collection of the data and avoided possible concerns that a matched-sample approach might lead to biases in unobserved characteristics between the groups presented with each preference-elicitation method. The order of the choice tasks was randomised across individuals to avoid bias. The empirical analysis undertaken on the BWS and DCE data revealed that the two methods produced comparable results. Given that the data suggest these methods produce consistent results we can then consider the wider advantages and disadvantages of each method. These are discussed and the decision to take forward and further develop the best-worst scaling task for future phases of the research is explained. The second question, on the stability of preferences, can be addressed from data that we collected in two subsequent phases of data collection using best-worst scaling experiments. In each wave, 500 members of the general public were interviewed to establish their preference weights. These two waves of data collection were undertaken a year apart. Whilst the instrument was subjected to some minor improvements between the two years, this data offers some valuable insights in to the stability of population preferences that have been collected using a best-worst scaling instrument. This paper provides some useful empirical findings on these key questions, although it is acknowledged that the findings may in part be contingent on the context of the study. The paper provides useful insights for other researchers similarly interested in exploring alternative methods available for eliciting data capable of supporting the estimation of discrete choice models for valuation studies and also discusses the benefits that best-worst scaling tasks can provide for some applications.