Choice certainty in Discrete Choice Experiments: Will eye tracking provide useful measures?

In this study, we conduct a Discrete Choice Experiment (DCE) using eye tracking technology to investigate if eye movements during the completion of choice sets reveal information about respondents’ choice certainty. We hypothesise that the number of times that respondents shift their visual attention between the alternatives in a choice set reflects their stated choice certainty. Based on one of the largest samples of eye tracking data in a DCE to date, we find evidence in favor of our hypothesis. We also link eye tracking observations to model-based choice certainty through parameterization of the scale function in a random parameters logit model. We find that choices characterized by more frequent gaze shifting do indeed exhibit a higher degree of error variance, however, this effects is insignificant once response time is controlled for. Overall, findings suggest that eye tracking can provide an observable and exogenous variable indicative of choice certainty, potentially improving the handling of respondent certainty and thus the performance of the choice models in DCEs. However, in our empirical case the benefits of using eye movement data as a proxy for choice certainty in the choice model are small at best, and our results suggest that response time provides a better proxy for stated choice certainty and provides larger improvements in model performance.

[1]  M. Shadlen,et al.  Choice Certainty Is Informed by Both Evidence and Decision Time , 2014, Neuron.

[2]  David A. Hensher,et al.  The Mixed Logit Model: the State of Practice and Warnings for the Unwary , 2001 .

[3]  Alejandro Caparrós,et al.  Information and visual attention in contingent valuation and choice modeling: field and eye-tracking experiments applied to reforestations in Spain , 2015 .

[4]  Morten Raun Mørkbak,et al.  The first time is the hardest: A test of ordering effects in choice experiments , 2012 .

[5]  John B. Loomis,et al.  Alternative approaches for incorporating respondent uncertainty when estimating willingness to pay: the case of the Mexican spotted owl , 1998 .

[6]  J. Durbin,et al.  Testing for serial correlation in least squares regression. II. , 1950, Biometrika.

[7]  Minoru Nakayama,et al.  Estimation of certainty for responses to multiple-choice questionnaires using eye movements , 2008, TOMCCAP.

[8]  Stephane Hess,et al.  On the use of a Modified Latin Hypercube Sampling (MLHS) method in the estimation of a Mixed Logit Model for vehicle choice , 2006 .

[9]  Leif Mattsson,et al.  Discrete choice under preference uncertainty: an improved structural model for contingent valuation. , 1995 .

[10]  Gregory L. Poe,et al.  Elicitation Effects in Contingent Valuation: Comparisons to a Multiple Bounded Discrete Choice Approach , 1998 .

[11]  W. Michael Hanemann,et al.  Welfare Evaluations in Contingent Valuation Experiments with Discrete Responses , 1984 .

[12]  Søren Bøye Olsen,et al.  Handling respondent uncertainty in Choice Experiments: Evaluating recoding approaches against explicit modelling of uncertainty , 2009 .

[13]  R. Dolan,et al.  Confidence in value-based choice , 2012, Nature Neuroscience.

[14]  Tobias Börger,et al.  Are Fast Responses More Random? Testing the Effect of Response Time on Scale in an Online Choice Experiment , 2015, Environmental and Resource Economics.

[15]  A. Rangel,et al.  Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions , 2011, Proceedings of the National Academy of Sciences.

[16]  Timothy J. Pleskac,et al.  Two-stage dynamic signal detection: a theory of choice, decision time, and confidence. , 2010, Psychological review.

[17]  Eyal M. Reingold,et al.  Eye Movement Monitoring as a Process Tracing Methodology in Decision Making Research , 2011 .

[18]  J. Bennett,et al.  Preference uncertainty in stated preference studies: facts and artefacts , 2013 .

[19]  K. Rayner,et al.  Measuring word recognition in reading: eye movements and event-related potentials , 2003, Trends in Cognitive Sciences.

[20]  John M. Rose,et al.  Consistently inconsistent: The role of certainty, acceptability and scale in choice , 2013 .

[21]  Walter R. Boot,et al.  Detecting Transient Changes in Dynamic Displays: The More You Look, the Less You See , 2006, Hum. Factors.

[22]  Yves Croissant,et al.  Panel data econometrics in R: The plm package , 2008 .

[23]  Wim Verbeke,et al.  Sustainability labels on coffee: Consumer preferences, willingness-to-pay and visual attention to attributes , 2015 .

[24]  Ta Theo Arentze,et al.  Transport stated choice responses: effects of task complexity, presentation format and literacy , 2003 .

[25]  Ian Krajbich,et al.  Visual fixations and the computation and comparison of value in simple choice , 2010, Nature Neuroscience.

[26]  Vincenzina Caputo,et al.  Consumers’ willingness to pay for organic chicken breast: Evidence from choice experiment , 2011 .

[27]  Jordan J. Louviere,et al.  Using facial EMG and eye tracking to study integral affect in discrete choice experiments , 2015 .

[28]  M. Arellano,et al.  Computing Robust Standard Errors for Within-Groups Estimators , 2009 .

[29]  Jeffrey M. Wooldridge,et al.  Introductory Econometrics: A Modern Approach , 1999 .

[30]  Michel Bierlaire,et al.  BIOGEME: a free package for the estimation of discrete choice models , 2003 .

[31]  Bodo Steiner,et al.  Personal Values and Decision Making: Evidence from Environmental Footprint Labeling in Canada , 2013 .

[32]  Richard T. Carson,et al.  Experimental analysis of choice , 1991 .

[33]  David A. Hensher,et al.  Not bored yet – revisiting respondent fatigue in stated choice experiments , 2012 .

[34]  Stephane Hess,et al.  Decision uncertainty in multi-attribute stated preference studies , 2016 .

[35]  Jacob L. Orquin,et al.  Attention and choice: a review on eye movements in decision making. , 2013, Acta psychologica.

[36]  R. Ready,et al.  How Do Respondents with Uncertain Willingness to Pay Answer Contingent Valuation Questions? , 2001, Land Economics.

[37]  Roy Brouwer,et al.  Choice Certainty and Consistency in Repeated Choice Experiments , 2010 .

[38]  Stuart M. Speedie,et al.  The value of structured data elements from electronic health records for identifying subjects for primary care clinical trials , 2015, BMC Medical Informatics and Decision Making.

[39]  Melissa R. Beck,et al.  Uncertainty Compensation in Human Attention: Evidence from Response Times and Fixation Durations , 2010, PloS one.

[40]  John M. Rose,et al.  Can you ever be certain? Reducing hypothetical bias in stated choice experiments via respondent reported choice certainty , 2016 .

[41]  G. Cornelis van Kooten,et al.  Treating respondent uncertainty in contingent valuation: A comparison of empirical treatments , 2007 .

[42]  Kevin J. Boyle,et al.  Using Conjoint Analysis to Derive Estimates of Compensating Variation , 1996 .

[43]  P. Boxall,et al.  Complexity in Choice Experiments: Choice of the Status Quo Alternative and Implications for Welfare Measurement , 2009 .

[44]  J. Louviere,et al.  The Role of the Scale Parameter in the Estimation and Comparison of Multinomial Logit Models , 1993 .

[45]  T. Stevens,et al.  A comparison of alternative certainty calibration techniques in contingent valuation , 2006 .

[46]  D. McFadden.  Conditional logit analysis of qualitative choice behavior , 1972 .

[47]  S. Shimojo,et al.  Gaze bias both reflects and influences preference , 2003, Nature Neuroscience.

[48]  Thomas V. Wiecki,et al.  Eye tracking and pupillometry are indicators of dissociable latent decision processes. , 2014, Journal of experimental psychology. General.

[49]  Adrian Pagan,et al.  The Lagrange Multiplier Test and its Applications to Model Specification in Econometrics , 1980 .

[50]  C. Starmer,et al.  Preference Anomalies, Preference Elicitation and the Discovered Preference Hypothesis , 2005 .

[51]  C. Koch,et al.  Relative visual saliency differences induce sizable bias in consumer choice , 2012 .

[52]  D. Hensher,et al.  Stated Choice Methods: Analysis and Applications , 2000 .

[53]  Hua Wang,et al.  Treatment of “Don't-Know” Responses in Contingent Valuation Surveys: A Random Valuation Model , 1997 .

[54]  K. Train.  Discrete Choice Methods with Simulation , 2003 .

[55]  Christof Koch,et al.  The Drift Diffusion Model Can Account for the Accuracy and Reaction Time of Value-Based Choices Under High and Low Time Pressure , 2010, Judgment and Decision Making.

[56]  Frode Alfnes,et al.  Eliciting consumers' willingness to pay for organic and welfare-labelled salmon in a non-hypothetical choice experiment , 2010 .

[57]  Søren Bøye Olsen,et al.  Tough and Easy Choices: Testing the Influence of Utility Difference on Stated Certainty-in-Choice in Choice Experiments , 2011 .

[58]  Iain Fraser,et al.  Visual Attention and Attribute Attendance in Multi-Attribute Choice Experiments , 2015 .

[59]  Richard C. Bishop,et al.  Using Donation Mechanisms to Value Nonuse Benefits from Public Goods , 1997 .

[60]  Danny Campbell,et al.  Response time in online stated choice experiments: the non-triviality of identifying fast and slow respondents , 2017 .

[61]  Duncan Mortimer,et al.  Lost in the crowd? Using eye-tracking to investigate the effect of complexity on attribute non-attendance in discrete choice experiments , 2015, BMC Medical Informatics and Decision Making.

[62]  A. Daly,et al.  Use of the logit scaling approach to test for rank-order and fatigue effects in stated preference data , 1994 .