Internet CV surveys – a cheap, fast way to get large samples of biased values?

With the current growth in broadband penetration, Internet is likely to be the data collection mode of choice for contingent valuation (CV) and stated preference research in the not so distant future. However, little is known about how this survey mode may influence data quality and welfare estimates. In a controlled field experiment as part of a large national CV survey estimating willingness to pay (WTP) for biodiversity protection plans, we assign two groups sampled from the same panel of respondents, either to an Internet or in-person interview mode. Our design is better able than previous mode comparison studies to isolate measurement effects from sample composition effects. Looking in particular for indications of social desirability bias and satisficing (shortcutting the response process) we find little evidence in our data. We find that the extent of “don’t know”, zeros and protest responses to the WTP question (with a payment card) is very similar between modes. Mean WTP is somewhat higher in the interview sample, though we cannot reject equality on the 10 percent level. We also consider equivalence, i.e. whether the WTP difference is larger than a practically trivial predetermined bound. We can reject that the difference is larger than 30 percent, but fail to reject an equivalency bound of 20 percent on the 10 percent level. Results are quite encouraging for the use of Internet as values do not seem to be significantly different or substantially biased compared to in-person interviews.

[1]  Henrik Lindhjem,et al.  Asking for Individual or Household Willingness to Pay for Environmental Goods? , 2008 .

[2]  Ian J. Bateman,et al.  Learning design contingent valuation (LDCV): NOAA guidelines, preference learning and coherent arbitrariness , 2008 .

[3]  David Sanders,et al.  Does Mode Matter For Modeling Political Choice? Evidence From the 2005 British Election Study , 2007, Political Analysis.

[4]  Jon A. Krosnick,et al.  The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples , 2007, Political Analysis.

[5]  P. Chisnall Mail and Internet Surveys: The Tailored Design Method , 2007, Journal of Advertising Research.

[6]  Tiago Domingos,et al.  Testing for the survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews , 2007 .

[7]  Don A Dillman,et al.  Design effects in the transition to web-based surveys. , 2007, American journal of preventive medicine.

[8]  D. Macmillan,et al.  Contingent Valuation: Comparing Participant Performance in Group-Based Approaches and Personal Interviews , 2007, Environmental Values.

[9]  P. Ubel,et al.  An alternative approach for eliciting willingness-to-pay: A randomized Internet trial , 2007, Judgment and Decision Making.

[10]  F. Conrad,et al.  Color, Labels, and Interpretive Heuristics for Response Scales , 2007 .

[11]  Robert J. Johnston,et al.  Choice experiments, site similarity and benefits transfer , 2007 .

[12]  Henrik Lindhjem,et al.  20 Years of Stated Preference Valuation of Non-Timber Benefits from Fennoscandian Forests: A Meta-Analysis , 2007 .

[13]  N. Hanley,et al.  Contingent valuation: Environmental polling or preference engine? , 2006 .

[14]  I. Hodge,et al.  Participatory Environmental Education and Willingness to Pay for River Basin Management: Empirical Evidence from Nigeria , 2006, Land Economics.

[15]  Peter Lynn,et al.  Telephone versus face-to-face interviewing: mode effects on data quality and likely causes: report on phase II of the ESS-Gallup mixed mode methodology project , 2006 .

[16]  Alan Krupnick,et al.  Valuation of Natural Resource Improvements in the Adirondacks , 2006, Land Economics.

[17]  Martyn Denscombe,et al.  Web-Based Questionnaires and the Mode Effect , 2006 .

[18]  D. Schwappach,et al.  "Quick and dirty numbers"? The reliability of a stated-preference technique for the measurement of preferences for resource allocation. , 2006, Journal of health economics.

[19]  J. Loomis,et al.  Testing the Convergent Validity of Videotape Survey Administration and Phone Interviews in Contingent Valuation , 2006 .

[20]  H. Thurston Non-market Valuation on the Internet , 2006 .

[21]  Mick P. Couper,et al.  Technology Trends in Survey Data Collection , 2005 .

[22]  Scott Fricker,et al.  An Experimental Comparison of Web and Telephone Surveys , 2005 .

[23]  Carol L. Silva,et al.  Testing for Budget Constraint Effects in a National Advisory Referendum Survey on the Kyoto Protocol , 2005 .

[24]  David Brockington,et al.  Social Desirability and Response Validity: A Comparative Analysis of Overreporting Voter Turnout in Five Countries , 2005 .

[25]  Robert J. Johnston,et al.  Systematic Variation in Willingness to Pay for Aquatic Resource Improvements and Implications for Benefit Transfer: A Meta-Analysis , 2005 .

[26]  Shawn A. Ross,et al.  Survey Methodology , 2005, The SAGE Encyclopedia of the Sociology of Religion.

[27]  D. Kristófersson,et al.  Validity Tests of Benefit Transfer – Are We Performing the Wrong Tests? , 2005 .

[28]  F. Conrad,et al.  Spacing, Position, and Order Interpretive Heuristics for Visual Features of Survey Questions , 2004 .

[29]  Thomas Laitila,et al.  Economic Valuation with Stated Preference Techniques: A Manual , 2004 .

[30]  David A. Harpman,et al.  Unit Non-Response Bias in the Interval Data Model , 2004, Land Economics.

[31]  I. Bateman,et al.  First impressions count: interviewer appearance and information effects in stated preference studies , 2004 .

[32]  Jennifer Davis Assessing Community Preferences for Development Projects: Are Willingness-to-Pay Studies Robust to Mode Effects? , 2004 .

[33]  Timothy C. Haab,et al.  Telephone presurveys, self-selection, and non-response bias to mail and Internet surveys in economic research , 2004 .

[34]  Robert P. Berrens,et al.  Information and effort in contingent valuation surveys: application to global climate change using national internet samples ☆ , 2004 .

[35]  Thomas Holtgraves,et al.  Social Desirability and Self-Reports: Testing Models of Socially Desirable Responding , 2004, Personality & social psychology bulletin.

[36]  Robert Cameron Mitchell,et al.  Social Desirability Bias in Contingent Valuation Surveys Administered Through In-Person Interviews , 2003, Land Economics.

[37]  David Laibson,et al.  The Allocation of Attention: Theory and Evidence , 2003 .

[38]  Melanie C. Green,et al.  Telephone versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Response Bias , 2003 .

[39]  Robert P. Berrens,et al.  The Advent of Internet Surveys for Political Research: A Comparison of Telephone and Internet Samples , 2003, Political Analysis.

[40]  R. Kramer,et al.  Estimating the economic value of water quality protection in the Catawba River basin , 2002 .

[41]  J. R. DeShazo,et al.  Designing Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency , 2002 .

[42]  Timothy C. Haab,et al.  Valuing Environmental and Natural Resources: The Econometrics of Non-Market Valuation , 2002 .

[43]  M. Larsen,et al.  The Psychology of Survey Response , 2002 .

[44]  K. Boyle,et al.  Doubt, Doubts, and Doubters: The Genesis of a New Research Agenda? , 2001 .

[45]  J. Epstein,et al.  Insuring sample equivalence across internet and paper-and-pencil assessments , 2001, Comput. Hum. Behav..

[46]  A. Tversky,et al.  Choices, Values, and Frames , 2000 .

[47]  V. Smith,et al.  JEEM and Non-market Valuation: 1974–1998 , 2000 .

[48]  Christopher Cornwell,et al.  Survey Response‐Related Biases in Contingent Valuation: Concepts, Remedies, and Empirical Application to Valuing Aquatic Plant Management , 2000 .

[49]  John B. Loomis,et al.  Using meta‐analysis for benefit transfer: In‐sample convergent validity tests of an outdoor recreation database , 2000 .

[50]  M. Couper A REVIEW OF ISSUES AND APPROACHES , 2000 .

[51]  John W. Payne,et al.  Measuring Constructed Preferences: Towards a Building Code , 1999 .

[52]  D. Kahneman,et al.  Does Living in California Make People Happy? A Focusing Illusion in Judgments of Life Satisfaction , 1998 .

[53]  J. Stanton AN EMPIRICAL ASSESSMENT OF DATA COLLECTION USING THE INTERNET , 1998 .

[54]  J. Vaske,et al.  Mail versus Telephone Surveys: Potential Biases in Expenditure and Willingness-to-Pay Data , 1998 .

[55]  Glenn C. Blomquist,et al.  Resource Quality Information and Validity of Willingness to Pay in Contingent Valuation , 1998 .

[56]  S. Navrud,et al.  Environmental Valuation – To Use or Not to Use? A Comparative Study of the United States and Europe , 1997 .

[57]  R. Berrens,et al.  Contingent valuation of rural tourism development with tests of scope and mode stability. , 1997 .

[58]  John B. Loomis,et al.  Comparison of Mail and Telephone-Mail Contingent Valuation Surveys , 1994 .

[59]  J. L. Rogers,et al.  Using significance tests to evaluate equivalence between two experimental groups. , 1993, Psychological bulletin.

[60]  Dale Whittington,et al.  Giving Respondents Time to Think in Contingent Valuation Studies: A Developing Country Application* , 1992 .

[61]  J. Krosnick Response strategies for coping with the cognitive demands of attitude measures in surveys , 1991 .

[62]  J. Loomis,et al.  Evaluation of Mail and In-person Contingent Value Surveys: Results of a Study of Recreational Boaters , 1991 .

[63]  J. Andreoni IMPURE ALTRUISM AND DONATIONS TO PUBLIC GOODS: A THEORY OF WARM-GLOW GIVING* , 1990 .

[64]  Trudy Ann Cameron,et al.  OLS versus ML estimation of non-market resource values with payment card interval data , 1989 .

[65]  Nick Hanley,et al.  Valuing rural recreation benefits: an empirical comparison of two approaches. , 1989 .

[66]  Robert Cameron Mitchell,et al.  Using Surveys to Value Public Goods: The Contingent Valuation Method , 1989 .

[67]  Eugene B. Brody,et al.  A Psychological Perspective , 1986 .

[68]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[69]  Dadi Kristofersson,et al.  Can use and Non-use Values Be Transferred Across Countries? , 2007 .

[70]  Mark Dickiea,et al.  Valuation of Non-Market Goods Using Computer-Assisted Surveys: A Comparison of Data Quality from Internet and RDD Samples , 2007 .

[71]  J. Shogren Experimental Methods and Valuation , 2005 .

[72]  R. A. Rlinghaus Management and Ecological Note Testing the reliability and construct validity of a simple and inexpensive procedure to measure the use value of recreational fishing , 2004 .

[73]  Ian J. Bateman,et al.  On visible choice sets and scope sensitivity , 2004 .

[74]  Toyoaki Washida,et al.  Economic valuation of the Seto Inland Sea by using an Internet CV survey. , 2003, Marine pollution bulletin.

[75]  M. Loureiro,et al.  Interviewer Effects on the Valuation of Goods with Ethical and Environmental Attributes , 2003 .

[76]  Kevin J. Boyle,et al.  Contingent Valuation in Practice , 2003 .

[77]  Thomas C. Brown,et al.  A primer on nonmarket valuation , 2003 .

[78]  I. Bateman Economic valuation with stated preference techniques : a manual : department for transport , 2002 .

[79]  Melanie C. Green,et al.  The Survey Response Process in Telephone and Face-to-Face Surveys : Differences in Respondent Satisficing and Social Desirability Response Bias , 2001 .

[80]  Ian J. Bateman,et al.  Valuing Environmental Preferences: Theory and Practice of the Contingent Valuation Method in the US, EU, and developing Countries , 2001 .

[81]  Daniel McFadden,et al.  Rationality for Economists? , 1999 .

[82]  J. Bennett,et al.  Yea-Saying in Contingent Valuation Surveys , 1999 .

[83]  Gregory L. Poe,et al.  A Comparison of Hypothetical Phone and Mail Contingent Valuation Responses for Green-Pricing Electricity Programs , 1997 .

[84]  David J. Bjornstad,et al.  The contingent valuation of environmental resources: methodological issues and research needs. , 1996 .

[85]  Edward E. Leamer,et al.  Report of the NOOA Panel on Contingent Valuation , 1993 .

[86]  S. F. Edwards,et al.  Overlooked Biases in Contingent Valuation Surveys: Some Considerations , 1987 .

[87]  J. Heckman Sample selection bias as a specification error , 1979 .