Measurement, generalization, and publication: Sources of error in benefit transfers and their management

Abstract Convergent validity tests of benefit transfer accuracy show errors to range from a few percentage points to 100% and more. This paper discusses three potential sources of errors that affect the accuracy of benefit transfers. (1) The measurement of values is subject to random errors and the caprices that arise from the many judgments and technical assumptions required by the researchers who conduct the primary studies. Measurement error occurs when researchers' decisions affect the transferability of measures of value or as the result of sampling. (2) Generalization error occurs when a measure of value is generalized to unstudied sites or resources. Generalization error is inversely related to the correspondence between study sites and policy sites. (3) Publication selection bias occurs when the objectives for publishing research limit benefit transfer applications of research outcomes. Criteria for selecting which research results are published may be at odds with the needs of benefit transfer practitioners. Several means for overcoming these sources of error are offered: standardized application of tested methods; closer adherence to benefit transfer protocol; the establishment of an e-journal with explicit criteria for fully recording, reporting, and disseminating research, which has the primary objective of estimating empirical measures of value.

[1]  Roy Brouwer,et al.  The Validity of Environmental Benefits Transfer: Further Empirical Testing , 1999 .

[2]  Subhrendu K. Pattanayak,et al.  Is Meta-Analysis a Noah's Ark for Non-Market Valuation? , 2002 .

[3]  Mary Jo Kealy,et al.  Benefits Transfer in a Random Utility Model of Recreation , 1994 .

[4]  Tommy Stanley,et al.  Beyond Publication Bias , 2005 .

[5]  J. Zelmer.  Linear Public Goods Experiments: A Meta-Analysis , 2003 .

[6]  J. Poot,et al.  The Last Word on the Wage Curve? , 2005 .

[7]  Brett Day,et al.  Benefit Transfer in Europe: How Reliable Are Transfers between Countries? , 2004 .

[8]  Jeffrey Bennett,et al.  Choice Modelling, Non-Use Values and Benefit Transfer , 2000 .

[9]  E. Strobl,et al.  Multinational Companies and Productivity Spillovers: A Meta‐Analysis , 2001 .

[10]  S. Engel.  Benefit function transfer versus meta-analysis as policy-making tools: a comparison , 2002 .

[11]  Roy Brouwer,et al.  Environmental value transfer: state of the art and future prospects , 2000 .

[12]  John B. Loomis,et al.  Testing a meta-analysis model for benefit transfer in international outdoor recreation , 2001 .

[13]  Kevin J. Boyle,et al.  Benefit transfer studies: Myths, pragmatism, and idealism , 1992 .

[14]  S. Swallow,et al.  Context-Sensitive Benefit Transfer Using Stated Choice Models: Specification and Convergent Validity for Policy Analysis , 2005 .

[15]  Kristin Magnussen,et al.  Benefit transfer: testing for accuracy and reliability , 2002 .

[16]  R. Florax,et al.  A Meta-Analysis of Beta- Convergence , 2005 .

[17]  Bonnie G. Colby,et al.  Evaluating the Performance of Benefit Transfer: An Empirical Inquiry , 1997 .

[18]  J. M. Bland.  Discussion of the paper by Begg and Berlin , 1988 .

[19]  Teofilo Ozuna,et al.  Testing the Reliability of the Benefit Function Transfer Approach , 1996 .

[20]  Peter Nijkamp,et al.  Comparative Environmental Economic Assessment , 2002 .

[21]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[22]  J. Bowker,et al.  BENEFITS TRANSFER AND COUNT DATA TRAVEL COST MODELS: AN APPLICATION AND TEST OF A VARYING PARAMETER APPROACH WITH GUIDED WHITEWATER RAFTING , 1997 .

[23]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[24]  Hristos Doucouliagos,et al.  What Do Unions Do to Productivity? A Meta-Analysis , 2003 .

[25]  R. Florax,et al.  Methodological pitfalls in meta-analysis: publication bias , 2001 .

[26]  V. Kerry Smith,et al.  What have we learned since hotelling's letter?: A meta-analysis , 1990 .

[27]  David Neumark,et al.  Wheat from Chaff: Meta-Analysis as Quantitative Literature Review: Reply , 2002 .

[28]  Gregory L. Poe,et al.  Assessing the Accuracy of Benefits Transfers: Evidence from a Multi-State Contigent Valuation Study of Groundwater Quality , 1995 .

[29]  V. Kerry Smith,et al.  Signals or Noise? Explaining the Variation in Recreation Benefit Estimates , 1990 .

[30]  Peter Nijkamp,et al.  Price and Income Elasticities of Residential Water Demand: A Meta-Analysis , 2003, Land Economics.

[31]  Randall S. Rosenberger,et al.  SITE CORRESPONDENCE EFFECTS IN BENEFIT TRANSFERS: A META-ANALYSIS TRANSFER FUNCTION , 2002 .

[32]  G. Cornelis van Kooten,et al.  How costly are carbon offsets? A meta-analysis of carbon forest sinks , 2004 .

[33]  Hristos Doucouliagos,et al.  Publication Bias in the Economic Freedom and Economic Growth Literature , 2005 .

[34]  Yong-Suhk Wui,et al.  The economic value of wetland services: a meta-analysis , 2001 .

[35]  W. Martin,et al.  Evaluating the accuracy of the benefit transfer method: a rural water supply application in the USA. , 2001, Journal of environmental management.

[36]  Ståle Navrud,et al.  Environmental value transfer : issues and methods , 2007 .

[37]  Sudip Chattopadhyay,et al.  A Repeated Sampling Technique in Assessing the Validity of Benefit Transfer in Valuing Non-Market Goods , 2003, Land Economics.

[38]  T. Haab,et al.  THE ECONOMIC VALUE OF MARINE RECREATIONAL FISHING: APPLYING BENEFIT TRANSFER TO MARINE RECREATIONAL FISHERIES STATISTICS SURVEY (MRFSS) , 2004 .

[39]  Robert J. Johnston,et al.  Modeling relationships between use and nonuse values for surface water quality: A meta‐analysis , 2003 .

[40]  T. Sterling.  Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa , 1959 .

[41]  Stephen B. Jarrell,et al.  Meta-Regression Analysis: A Quantitative Method of Literature Surveys , 1989 .

[42]  Peter Nijkamp,et al.  THE LAST WORD ON THE WAGE CURVE? A META-ANALYTIC ASSESSMENT , 2003 .

[43]  Anne Rozan,et al.  Benefit Transfer: A Comparison of WTP for Air Quality between France and Germany , 2004 .

[44]  John B. Loomis,et al.  Using meta‐analysis for benefit transfer: In‐sample convergent validity tests of an outdoor recreation database , 2000 .

[45]  C. Begg,et al.  Publication bias : a problem in interpreting medical data , 1988 .

[46]  John B. Loomis,et al.  Testing Transferability of Recreation Demand Models Across Regions: A Study of Corps of Engineer Reservoirs , 1995 .

[47]  John B. Loomis,et al.  The Evolution of a More Rigorous Approach to Benefit Transfer: Benefit Function Transfer , 1992 .

[48]  R. Mooij,et al.  Meta-analysis: A Tool for Upgrading Inputs of Macroeconomic Policy Models , 2002 .

[49]  David Card,et al.  Time-Series Minimum-Wage Studies: A Meta-analysis , 1995 .

[50]  W. Hanemann,et al.  Adaptation and Its Measurement , 2000 .

[51]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[52]  John A List,et al.  Cigarette demand: a meta-analysis of elasticities. , 2003, Health economics.

[53]  George R. Parsons,et al.  Benefits transfer: conceptual problems in estimating water quality benefits using existing studies , 1992 .

[54]  Tommy Stanley,et al.  Meta-Regression Methods for Detecting and Estimating Empirical Effects in the Presence of Publication Selection , 2007 .

[55]  Randall S. Rosenberger,et al.  Correspondence and Convergence in Benefit Transfer Accuracy: Meta-Analytic Review of the Literature , 2007 .

[56]  Ian J. Bateman,et al.  Contrasting conventional with multi-level modelling approaches to meta-analysis: expectation consistency in UK woodland recreation values. , 2003 .

[57]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[58]  D. Kristófersson,et al.  Validity Tests of Benefit Transfer – Are We Performing the Wrong Tests? , 2005 .

[59]  V. Smith,et al.  Can Markets Value Air Quality? A Meta-Analysis of Hedonic Property Value Models , 1995, Journal of Political Economy.

[60]  Joel Huber,et al.  The Economic Value of Water Quality , 2008 .