Foretelling What Makes People Pay: Predicting the Results of Field Experiments on TV Fee Enforcement

One of the current challenges in field experimentation is creating an efficient design including individual treatments. Ideally, a pilot should be run in advance, but when a pilot is not feasible, any information about the effectiveness of potential treatments' to researchers is highly valuable. We run a laboratory experiment in which we forecast results of two large-scale field experiments focused on TV license fee collection to evaluate the extent to which it is possible to predict field experiment results using a non-expert subject pool. Our main result is that forecasters were relatively conservative regarding the absolute effectiveness of the treatments, but in most cases they correctly predicted the relative effectiveness. Our results suggest that, despite the artificiality of laboratory environments, forecasts generated there may provide valuable estimates of the effectiveness of treatments.

[1]  Matthias Heinz,et al.  Measuring Indirect Effects of Unfair Employer Behavior on Worker Productivity - A Field Experiment , 2017, SSRN Electronic Journal.

[2]  Lucas C. Coffman,et al.  Pathways of persuasion , 2020, Games Econ. Behav..

[3]  Ben Greiner,et al.  Subject pool recruitment procedures: organizing experiments with ORSEE , 2015, Journal of the Economic Science Association.

[4]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[5]  Devin G. Pope,et al.  Stability of Experimental Results: Forecasts and Evidence , 2019, SSRN Electronic Journal.

[6]  A. H. Murphy,et al.  “Good” Probability Assessors , 1968 .

[7]  Brian A. Nosek,et al.  Predicting replication outcomes in the Many Labs 2 study , 2018, Journal of Economic Psychology.

[8]  Gideon Nave,et al.  Evaluating replicability of laboratory experiments in economics , 2016, Science.

[9]  D. McKenzie,et al.  The impact of soft skills training on female youth employment: evidence from a randomized experiment in Jordan , 2016 .

[10]  A. Samek,et al.  Do Thank-You Calls Increase Charitable Giving? Expert Forecasts and Field Experimental Evidence , 2019, SSRN Electronic Journal.

[11]  Brian A. Nosek,et al.  Using prediction markets to estimate the reproducibility of scientific research , 2015, Proceedings of the National Academy of Sciences.

[12]  Alvin E. Roth,et al.  A choice prediction competition: Choices from experience and from description , 2010 .

[13]  L. V. Williams,et al.  Prediction Markets , 2003 .

[14]  Stefano DellaVigna,et al.  Predicting Experimental Results: Who Knows What? , 2016, Journal of Political Economy.

[15]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[16]  Rupert Sausgruber,et al.  TESTING ENFORCEMENT STRATEGIES IN THE FIELD: THREAT, MORAL APPEAL AND SOCIAL INFORMATION , 2013 .

[17]  U. Fischbacher.  z-Tree: Zurich toolbox for ready-made economic experiments , 1999 .

[18]  Devin G. Pope,et al.  What Motivates Effort? Evidence and Expert Forecasts , 2016 .

[19]  Lars Hornuf,et al.  Are sustainability-oriented investors different? Evidence from equity crowdfunding , 2021, The Journal of Technology Transfer.

[20]  Agnar Sandmo,et al.  Income tax evasion: a theoretical analysis , 1972 .