Reconceptualising the External Validity of Discrete Choice Experiments

External validity is a crucial but under-researched topic when considering using discrete choice experiment (DCE) results to inform decision making in clinical, commercial or policy contexts. We present the theory and tests traditionally used to explore external validity that focus on a comparison of final outcomes and review how this traditional definition has been empirically tested in health economics and other sectors (such as transport, environment and marketing) in which DCE methods are applied. While an important component, we argue that the investigation of external validity should be much broader than a comparison of final outcomes. In doing so, we introduce a new and more comprehensive conceptualisation of external validity, closely linked to process validity, that moves us from the simple characterisation of a model as being or not being externally valid on the basis of predictive performance, to the concept that external validity should be an objective pursued from the initial conceptualisation and design of any DCE. We discuss how such a broader definition of external validity can be fruitfully used and suggest innovative ways in which it can be explored in practice.

[1]  M Johannesson,et al.  Hypothetical versus real willingness to pay in the health care sector: results from a field experiment. , 2001, Journal of health economics.

[2]  Richard T. Carson,et al.  Contingent Valuation: A Comprehensive Bibliography and History , 2012 .

[3]  W. Adamowicz,et al.  Are Food Choices Really Habitual? Integrating Habits, Variety‐Seeking, and Compensatory Choice in a Utility‐Maximizing Framework , 2013 .

[4]  Arne Risa Hole,et al.  A discrete choice model with endogenous attribute attendance , 2011 .

[5]  D. McFadden,et al.  Suit the action to the word, the word to the action: Hypothetical choices and real decisions in Medicare Part D. , 2013, Journal of health economics.

[6]  Mandy Ryan,et al.  'Irrational' stated preferences: a quantitative and qualitative investigation. , 2005, Health economics.

[7]  Rosalie Viney,et al.  Empirical investigation of experimental design properties of discrete choice experiments in health care. , 2005, Health economics.

[8]  A. Daly,et al.  Handbook of Choice Modelling , 2014 .

[9]  Wagner A. Kamakura,et al.  A Multi-trait Multi-Method Validity Test of Partworth Estimates , 2007 .

[10]  Peter Martinsson,et al.  Do Hypothetical and Actual Marginal Willingness to Pay Differ in Choice Experiments?: Application to the Valuation of the Environment , 2001 .

[11]  Deborah Marshall,et al.  Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. , 2013, Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research.

[12]  Joanna Coast,et al.  Developing attributes and levels for discrete choice experiments using qualitative methods , 2007, Journal of health services research & policy.

[13]  Jordan J. Louviere,et al.  Perceptions versus Objective Measures of Environmental Quality in Combined Revealed and Stated Preference Models of Environmental Valuation , 1997 .

[14]  K. Lancaster.  A New Approach to Consumer Theory , 1966, Journal of Political Economy.

[15]  J. Brazier,et al.  Using a discrete choice experiment to estimate health state utility values. , 2012, Journal of health economics.

[16]  Tara Symonds,et al.  Using DCE and ranking data to estimate cardinal values for health states for deriving a preference-based single index from the sexual quality of life questionnaire. , 2009, Health economics.

[17]  Ann Netten,et al.  Using discrete choice experiments to estimate a preference-based measure of outcome--an application to social care for older people. , 2006, Journal of health economics.

[18]  Jorge E. Araña,et al.  Dynamic hypothetical bias in discrete choice experiments: Evidence from measuring the impact of corporate social responsibility on consumers demand☆ , 2013 .

[19]  Martin Natter,et al.  Real world performance of choice-based conjoint models , 2002, Eur. J. Oper. Res..

[20]  I. Bateman.  Economic valuation with stated preference techniques : a manual : department for transport , 2002 .

[21]  J. Louviere,et al.  Using discrete choice experiments to investigate subject preferences for preventive asthma medication , 2007, Respirology.

[22]  M. Ben-Akiva,et al.  EMPIRICAL TEST OF A CONSTRAINED CHOICE DISCRETE MODEL : MODE CHOICE IN SAO PAULO, BRAZIL , 1987 .

[23]  Richard T. Carson,et al.  The discrete choice experiment approach to environmental contingent valuation , 2014 .

[24]  Gregory L. Poe,et al.  Alternative Nonmarket Value-elicitation Methods: Are the Underlying Preferences the Same? Alternative Nonmarket Value-elicitation Methods: Are the Underlying Preferences the Same? , 1999 .

[25]  John Humphreys,et al.  Junior doctors' preferences for specialty choice. , 2012, Journal of health economics.

[26]  Joffre Swait,et al.  Using stated preference and revealed preference modeling to evaluate prescribing decisions. , 2004, Health economics.

[27]  C. A. Kennedy.  Revealed preference valuation compared to contingent valuation: radon-induced lung cancer prevention. , 2002, Health economics.

[28]  C. Manski.  The structure of random utility models , 1977 .

[29]  R. Lucas.  Econometric policy evaluation: A critique , 1976 .

[30]  Mark D. Uncles,et al.  Discrete Choice Analysis: Theory and Application to Travel Demand , 1987 .

[31]  D. Hensher,et al.  Stated Choice Methods: Analysis and Applications , 2000 .

[32]  Joffre Swait,et al.  A NON-COMPENSATORY CHOICE MODEL INCORPORATING ATTRIBUTE CUTOFFS , 2001 .

[33]  C. Donaldson,et al.  Discrete choice experiments in health economics , 2005, The European Journal of Health Economics.

[34]  M. Ryan,et al.  Comparing welfare estimates from payment card contingent valuation and discrete choice experiments. , 2009, Health economics.

[35]  Moshe Ben-Akiva,et al.  Incorporating random constraints in discrete models of choice set generation , 1987 .

[36]  J. Louviere,et al.  Conducting Discrete Choice Experiments to Inform Healthcare Decision Making , 2012, PharmacoEconomics.

[37]  C. Jenkins,et al.  Patient preferences for managing asthma: results from a discrete choice experiment. , 2007, Health economics.

[38]  Andreas Herrmann,et al.  Conjoint Measurement: Methods and Applications , 2000 .

[39]  Terry Elrod,et al.  A new integrated model of noncompensatory and compensatory decision strategies , 2004 .

[40]  S. Watson,et al.  Understanding the consequences of consequentiality: Testing the validity of stated preferences in the field , 2012 .

[41]  Kenneth A. Small,et al.  Valuing time and reliability: assessing the evidence from road pricing demonstrations , 2003 .

[42]  Mandy Ryan,et al.  Discrete choice experiments in health economics: a review of the literature. , 2012, Health economics.

[43]  Moshe Ben-Akiva,et al.  Discrete Choice Analysis: Theory and Application to Travel Demand , 1985 .

[44]  Joffre Swait,et al.  Using repeated cross-sectional travel surveys to enhance forecasting robustness: Accounting for changing mode preferences , 2014 .

[45]  Rick L. Andrews,et al.  Studying Consideration Effects in Empirical Choice Models Using Scanner Panel Data , 1995 .

[46]  Emily Lancsar,et al.  Choice experiments in health: the good, the bad, the ugly and toward a brighter future , 2009, Health Economics, Policy and Law.

[47]  D. Street,et al.  Decisions about Pap tests: what influences women and providers? , 2009, Social science & medicine.

[48]  Emily Lancsar,et al.  Deleting 'irrational' responses from discrete choice experiments: a case of investigating or imposing preferences? , 2006, Health economics.

[49]  Marisol Castro,et al.  Estimation of a constrained multinomial logit model , 2012, Transportation.

[50]  Larry Lockshin,et al.  Combining discrete choice and informed sensory testing in a two-stage process: can it predict wine market share? , 2010 .

[51]  John M. Rose,et al.  Hypothetical bias in Stated Choice Experiments: Is it a problem? And if so, how do we deal with it? , 2014 .

[52]  Rick L. Andrews,et al.  On the Recoverability of Choice Behaviors with Random Coefficients Choice Models in the Context of Limited Data and Unobserved Effects , 2008, Manag. Sci..

[53]  Jordan J. Louviere,et al.  Best-Worst Scaling: Theory, Methods and Applications , 2015 .

[54]  Vikki Entwistle,et al.  Rationalising the 'irrational': a think aloud study of discrete choice experiment responses. , 2009, Health economics.

[55]  D. Horsky,et al.  Stating Preference for the Ethereal but Choosing the Concrete: How the Tangibility of Attributes Affects Attribute Weighting in Value Elicitation and Choice , 2003 .

[56]  J. Louviere,et al.  The Role of the Scale Parameter in the Estimation and Comparison of Multinomial Logit Models , 1993 .

[57]  F. Norwood,et al.  Bridging the gap between laboratory experiments and naturally occurring markets: An inferred valuation method , 2009 .

[58]  M. Ryan,et al.  Deriving distributional weights for QALYs through discrete choice experiments. , 2011, Journal of health economics.

[59]  Rosalie Viney,et al.  Using stated preference discrete choice modelling to evaluate the introduction of varicella vaccination. , 2002, Health economics.