Experimentation and the evaluation of energy efficiency programs

The use of experiments— particularly randomized controlled trials (RCTs) where subjects are randomly assigned to treatment and control conditions—has rarely been applied to the process of improving the design of energy efficiency programs and, more fundamentally, to determining the net savings from energy efficiency programs. This paper discusses the use of experimentation in the energy efficiency program field with the hope of explaining how these experiments can be used, and identifying the barriers to their use will cause more experimentation to occur. First, a brief overview of experimental methods is presented. This discussion describes the advantages and disadvantages of conducting experiments in the context of the development and evaluation of energy efficiency programs. It then discusses barriers to the use of experimental methods (including cost and equity issues) and suggests some ways of overcoming these barriers. Finally, recommendations are made for implementing key social experiments, discussing the types of energy efficiency programs and issues that can make use of experimentation and variables that one might use for selecting treatments.

[1]  Michael Kremer,et al.  Chapter 61 Using Randomization in Development Economics Research: A Toolkit ★ , 2007 .

[2]  Edward Vine,et al.  Emerging evaluation issues: persistence, behavior, rebound, and policy , 2013 .

[3]  Carl Blumstein,et al.  Program evaluation and incentives for administrators of energy-efficiency programs: Can evaluation solve the principal/agent problem? , 2009 .

[4]  D. Campbell.  Reforms as experiments , 1969 .

[5]  Corinna Fischer.  Feedback on household electricity consumption: a tool for saving energy? , 2008 .

[6]  Karen Ehrhardt-Martinez,et al.  The Climate Imperative and Innovative Behavior: Encouraging Greater Advances in the Production of Energy-Efficient Technologies and Services , 2009 .

[7]  Sanem Sergici,et al.  The Impact of Informational Feedback on Energy Consumption -- A Survey of the Experimental Evidence , 2009 .

[8]  D. Campbell,et al.  Evolving Methods for Enhancing Validity@@@Methodology and Epistemology for Social Science: Selected Papers , 1990 .

[9]  Michael J. Sullivan.  Using Experiments to Foster Innovation and Improve the Effectiveness of Energy Efficiency Programs , 2009 .

[10]  Lori Megdal,et al.  Evaluating Media Campaign Effectiveness: Others Do it, Why Don't We? , 2006 .

[11]  W. Shadish,et al.  Social Experiments: Some Developments over the Past Fifteen Years , 1994 .

[12]  H. Allcott,et al.  Social Norms and Energy Conservation , 2011 .

[13]  David H. Greenberg,et al.  Digest of Social Experiments , 2004 .

[14]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[15]  William N. Dunn,et al.  The experimenting society , 1998 .

[16]  Edward Miguel,et al.  Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities, Guide to Replication of Miguel and Kremer (2004) , 2014 .

[17]  Ralph Prahl,et al.  Emerging issues in the evaluation of energy-efficiency programs: the US experience , 2012 .

[18]  D. Torgerson,et al.  Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials | Cabinet Office , 2012 .

[19]  I. Ayres,et al.  Evidence from Two Large Field Experiments that Peer Comparison Feedback Can Reduce Residential Energy Usage , 2009 .

[20]  D. Torgerson,et al.  Designing randomised trials in health, education and the social sciences : an introduction , 2008 .

[21]  Peter Cappers,et al.  An Analysis of the Effects of Residential Photovoltaic Energy Systems on Home Sales Prices in California , 2011 .

[22]  Paul C. Stern.  The Cambridge Handbook of Psychology and Economic Behaviour: Environmentally significant behavior in the home , 2008 .

[23]  Edward Vine,et al.  Energy Myth Ten – Energy Efficiency Measures are Unreliable, Unpredictable, and Unenforceable , 2007 .

[24]  Benjamin K. Sovacool,et al.  Energy and American Society: Thirteen Myths , 2007 .

[25]  Loren Lutzenhiser,et al.  Behavioral Assumptions Underlying California Residential Sector Energy Efficiency Programs , 2009 .

[26]  Sanem Sergici,et al.  Household Response to Dynamic Pricing of Electricity - A Survey of the Empirical Evidence , 2010 .

[27]  Karen Ehrhardt-Martinez,et al.  Pursuing Energy-Efficient Behavior in a Regulatory Environment: Motivating Policymakers, Program Administrators, and Program Implementers , 2009 .

[28]  Peter John,et al.  Mobilizing citizen effort to enhance environmental outcomes: a randomized controlled trial of a door-to-door recycling campaign. , 2009, Journal of environmental management.

[29]  Paul L. Joskow,et al.  Incentive Regulation For Electric Utilities , 1986 .

[30]  Edward Vine,et al.  Strategies and policies for improving energy efficiency programs : Closing the loop between evaluation and implementation , 2008 .