Demand-side management policy : mechanisms for success and failure

Demand-side management (DSM) refers to activities undertaken on the demand-side of energy meters that seek to meet energy policy objectives. The policy side of DSM has received limited attention in the academic literature, with previous research focussing on technological trials, utility programmes and modelling studies of the potential of DSM. Within the field of DSM policy evaluation, much of the work has concentrated on policy impacts rather than policy mechanisms. The thesis contributes to filling this research gap by determining the key factors for success and failure for various DSM policies and countries (and sub-national states). A global systematic review of the DSM policy evaluation evidence was conducted. The method included the critical appraisal of the quality of the evidence base and the final sample included 119 high quality documents (covering 690 evaluations) from 35 databases, covering 30 countries, 36 sub-national states, and 21 individual DSM policies and policy packages. A technique was developed to combine factor frequency and weighting analyses in order to establish the success and failure factors that were both frequent and highly weighted for given DSM policies and countries/states. Overall, across policies and countries/states, regulatory frameworks and appropriate incentives are the most crucial success factors, and a lack of monitoring and technical issues (primarily programme management issues) are the most crucial failure factors. California, China, the UK and the USA have experienced the greatest success with DSM policies, each having successfully implemented and evaluated 9-10 policies. Utility obligations, performance standards and alternative utility business models have been the most successful policies overall, whilst labelling, information campaigns, and loans and subsidies have been the least successful. However, all policies show examples of both success and failure in specific contexts and the research has identified which key factors cause various demand-side policies to succeed or fail.

[1]  Hamidreza Zareipour,et al.  Energy storage for mitigating the variability of renewable electricity sources: An updated review , 2010 .

[2]  David Crossley,et al.  INTERNATIONAL BEST PRACTICE IN USING ENERGY EFFICIENCY AND DEMAND MANAGEMENT TO SUPPORT ELECTRICITY NETWORKS , 2010 .

[3]  Lloyd Harrington,et al.  ENERGY LABELLING AND STANDARDS PROGRAMS THROUGHOUT THE WORLD , 2004 .

[4]  Peter Rumsey,et al.  Energy Conservation in Typical Asian Countries , 1997 .

[5]  J. Bernard,et al.  Hydro-Québec energy savings programs: ‘Watt’ are they worth? , 1996 .

[6]  A Laupacis,et al.  Assessing the quality of randomized trials: reliability of the Jadad scale. , 1999, Controlled clinical trials.

[7]  Lingfeng Wang,et al.  Smart meters for power grid — Challenges, issues, advantages and status , 2011, 2011 IEEE/PES Power Systems Conference and Exposition.

[8]  Paul Upham,et al.  Sustainability constraints on UK bioenergy development , 2009 .

[9]  S. Nadel,et al.  Utility DSM: Off the Coasts and into the Heartland , 2010 .

[10]  R. Likert “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[11]  Douglas C. Larson,et al.  Demand Response in the West: Lessons for States and Provinces , 2004 .

[12]  Brian W. Hogwood,et al.  Policy Analysis For The Real World , 1984 .

[13]  G. Bahgat Europe's energy security: challenges and opportunities , 2006 .

[14]  S. Schwartz,et al.  The right answer for the wrong question: consequences of type III error for public health research. , 1999, American journal of public health.

[15]  Stefan Muench,et al.  A systematic review of bioenergy life cycle assessments , 2013 .

[16]  Peter N. Posch,et al.  Teachers Investigate Their Work: An introduction to action research across the professions , 1993 .

[17]  Gideon Kruseman,et al.  Long-term price and environmental effects in a liberalised electricity market , 2008 .

[18]  Richard Rose,et al.  What is Lesson-Drawing? , 1991, Journal of Public Policy.

[19]  Richard Rose,et al.  Lesson-drawing in Public Policy: A Guide to Learning Across Time and Space , 1993 .

[20]  Frank Fischer,et al.  Evaluating Public Policy , 1995 .

[21]  Wouter Poortinga,et al.  Exploring the Dimensionality of Trust in Risk Regulation , 2003, Risk analysis : an official publication of the Society for Risk Analysis.

[22]  C. Gellings Before demand‐side management is discarded, let’s see what pieces should be kept , 2000 .

[23]  S. Sorrell Energy Substitution, Technical Change and Rebound Effects , 2014 .

[24]  Elisa Puzzolo,et al.  Enablers and Barriers to Large-Scale Uptake of Improved Solid Fuel Stoves: A Systematic Review , 2013, Environmental health perspectives.

[25]  P. John Making Policy Work , 2011 .

[26]  S. Owens,et al.  How to change attitudes and behaviours in the context of energy , 2008 .

[27]  Steve Sorrell,et al.  Reducing energy demand: A review of issues, challenges and approaches , 2015 .

[28]  Chia-Chin Cheng,et al.  Electricity Demand-Side Management for an Energy Efficient Future in China: Technology Options and Policy Priorities , 2005 .

[29]  K. El Bakari,et al.  Prospects of a virtual power plant to control a cluster of distributed generation and renewable energy sources , 2009, 2009 44th International Universities Power Engineering Conference (UPEC).

[30]  Nick Eyre,et al.  Energy company obligations to save energy in Italy, the UK and France: what have we learnt? , 2009 .

[31]  Paulo Costa,et al.  The “Smart Paradox”: Stimulate the deployment of smart grids with effective regulatory instruments , 2014 .

[32]  Public Benefit Funds , 2000 .

[33]  Luisa F. Cabeza,et al.  Investigating greenhouse challenge from growing trends of electricity consumption through home appliances in buildings , 2014 .

[34]  Stefan Thomas,et al.  What makes a good policy? : Guidance for assessing and implementing energy efficiency policies , 2013 .

[35]  Rolf Wüstenhagen,et al.  Social acceptance of renewable energy innovation: An introduction to the concept , 2007 .

[36]  R. Walawalkar,et al.  Evolution and current status of demand response (DR) in electricity markets: Insights from PJM and NYISO , 2010 .

[37]  Marcel Didden,et al.  Demand Side Management in a competitive European market: Who should be responsible for its implementation? , 2003 .

[38]  Douglas Cooke,et al.  Empowering Customer Choice in Electricity Markets , 2011 .

[39]  Mark Peplow,et al.  Social sciences suffer from severe publication bias , 2014, Nature.

[40]  Eric Hirst,et al.  The Financial and Physical Insurance Benefits of Price-Responsive Demand , 2002 .

[41]  Lorna A. Greening,et al.  Demand response resources: Who is responsible for implementation in a deregulated market? , 2010 .

[42]  Paul Ekins,et al.  The Macroeconomic Rebound Effect and the UK Economy , 2006 .

[43]  Lars J Nilsson,et al.  Theory-based policy evaluation of 20 energy efficiency instruments , 2008 .

[44]  Ray Pawson,et al.  Evidence-based Policy: In Search of a Method , 2002 .

[45]  S. Pachauri,et al.  Synergies between Energy Efficiency and Energy Access Policies and Strategies , 2012 .

[46]  L. B. Leopold,et al.  A procedure for evaluating environmental impact , 1971 .

[47]  Mehmet Suat Kayikci,et al.  The European Third Energy Package: How Significant for the Liberalisation of Energy Markets in the European Union? , 2011 .

[48]  George Huitema,et al.  The Neglected Consumer: The Case of the Smart Meter Rollout in the Netherlands , 2011 .

[49]  A. Larsen Spending to save : evaluation of the energy efficiency obligation in Denmark , 2013 .

[50]  M. Patterson What is energy efficiency?: Concepts, indicators and methodological issues , 1996 .

[51]  Lester Baxter,et al.  Net lost revenue adjustment (NLRA) mechanisms for utility DSM programs , 1995 .

[52]  Goran Strbac,et al.  Demand side management: Benefits and challenges ☆ , 2008 .

[53]  Yongzhen Yu,et al.  Policy redesign for solving the financial bottleneck in demand side management (DSM) in China , 2010 .

[54]  E. M. Meyers,et al.  Demand-Side Carbon Reduction Strategies in an Era of Electric Industry Competition , 1999 .

[55]  D. Crossley,et al.  Tradeable energy efficiency certificates in Australia , 2008 .

[56]  J. Torriti,et al.  Demand response experience in Europe: Policies, programmes and implementation , 2010 .

[57]  Aie Gadgets and Gigawatts : Policies for Energy Efficient Electronics , 2009 .

[58]  Diana Ürge-Vorsatz,et al.  Appraisal of policy instruments for reducing buildings' CO2 emissions , 2007 .

[59]  Eamon McKeogh,et al.  Techno-economic review of existing and new pumped hydro energy storage plant , 2010 .

[60]  N. Gurwick,et al.  A Systematic Review of Biochar Research, with a Focus on Its Stability in situ and Its Promise as a Climate Mitigation Strategy , 2013, PloS one.

[61]  Danish Energy Efficiency Policy: Revisited and Future Improvements , 2009 .

[62]  Sahin Albayrak,et al.  A simulation framework for smart meter security evaluation , 2011, 2011 IEEE International Conference on Smart Measurements of Future Grids (SMFG) Proceedings.

[63]  Pranoto Effendi,et al.  Political aspects of innovation: Examining renewable energy in Australia , 2012 .

[64]  Ray Pawson,et al.  Evidence-based policy: a realist perspective. , 2006 .

[65]  Paul M. Kellstedt,et al.  The Fundamentals of Political Science Research , 2008 .

[66]  Ernst Worrell,et al.  The German energy audit program for firms—a cost-effective way to improve energy efficiency? , 2012 .

[67]  N. Eyre,et al.  Lessons from energy efficiency policy and programmesin the UK from 1973 to 2013 , 2014 .

[68]  W. L. Lee,et al.  Regulatory and voluntary approaches for enhancing building energy efficiency , 2004 .

[69]  J Shepherd,et al.  The effectiveness and cost-effectiveness of behavioural interventions for the prevention of sexually transmitted infections in young people aged 13-19: a systematic review and economic evaluation. , 2010, Health technology assessment.

[70]  P. Linares,et al.  Energy Efficiency: Economics and Policy , 2010 .

[71]  Jeanne Daly,et al.  A hierarchy of evidence for assessing qualitative health research. , 2007, Journal of clinical epidemiology.

[72]  Jay Zarnikau,et al.  Demand participation in the restructured Electric Reliability Council of Texas market , 2010 .

[73]  P. Cappers,et al.  Demand Response in U.S. Electricity Markets: Empirical Evidence , 2010 .

[74]  Mark S. Granovetter The Strength of Weak Ties , 1973, American Journal of Sociology.

[75]  Mohammed H. Albadi,et al.  A summary of demand response in electricity markets , 2008 .

[76]  Diana Ürge-Vorsatz,et al.  Drivers of market transformation: analysis of the Hungarian lighting success story , 2001 .

[77]  K. Uma Rao,et al.  A novel predictive DSM strategy to match power outage pattern for optimal cost with solar and diesel power , 2014, 2014 IEEE Innovative Smart Grid Technologies - Asia (ISGT ASIA).

[78]  S. Sorrell,et al.  Improving the evidence base for energy policy: the role of systematic reviews , 2007 .

[79]  P. Bertoldi,et al.  Tradable white certificate schemes: fundamental concepts , 2008 .

[80]  Paul L. Joskow,et al.  Lessons Learned From Electricity Market Liberalization , 2008 .

[81]  Thomas O'Donoghue,et al.  Qualitative educational research in action : doing and reflecting , 2003 .

[82]  J. Cochran,et al.  Energy Efficiency Policy in the United States: Overview of Trends at Different Levels of Government , 2009 .

[83]  C. Sheinbaum,et al.  MEXICAN ELECTRIC END-USE EFFICIENCY: Experiences to Date , 1998 .

[84]  C.W. Gellings,et al.  The concept of demand-side management for electric utilities , 1985, Proceedings of the IEEE.

[85]  Tobias Fleiter,et al.  Efficiency of an Energy Audit Programme for SMEs in Germany - Results of an Evaluation Study , 2011 .

[86]  J. Steinberger,et al.  Profiting from negawatts: Reducing absolute consumption and emissions through a performance-based energy economy , 2009 .

[87]  D. Finon,et al.  The costs and benefits of white certificates schemes , 2012 .

[88]  Tim Jackson,et al.  Missing carbon reductions? Exploring rebound and backfire effects in UK households , 2010, Energy Policy.

[89]  Fereidoon P. Sioshansi,et al.  What Could Possibly Be Better than Real-Time Pricing? Demand Response , 2001 .

[90]  S. A. Onisko,et al.  Everyone wins! -- A program to upgrade energy efficiency in manufactured housing , 1994 .

[91]  Kito,et al.  Review of demand-side bidding programs: Impacts, costs, and cost-effectiveness , 1994 .

[92]  Christian Rehtanz,et al.  Balancing Renewable Electricity: Energy Storage, Demand Side Management, and Network Extension from an Interdisciplinary Perspective , 2012 .

[93]  A. Akobeng Understanding randomised controlled trials , 2005, Archives of Disease in Childhood.

[94]  Luisa F. Cabeza,et al.  Heating and cooling energy trends and drivers in buildings , 2015 .

[95]  D. Yanow Qualitative-interpretive methods in policy research , 2006 .

[96]  Gabrial Anandarajah,et al.  Implications of sustainability constraints on UK bioenergy development: Assessing optimistic and precautionary approaches with UK MARKAL , 2012 .

[97]  Philip E. T. Lewis,et al.  Research Methods for Business Students , 2006 .

[98]  Lorenzo Pagliano,et al.  Completing the Market for Least-Cost Energy Services , 2000 .

[99]  D. Bachrach Energy Efficiency Leadership in California: Preventing the Next Crisis , 2003 .

[100]  C. Clastres Smart grids: Another step towards competition, energy security and climate change objectives , 2011 .

[101]  J. Eto,et al.  Carrots and sticks: Shared-savings incentive programs for energy efficiency , 1990 .

[102]  M. Newborough,et al.  Demand-side management opportunities for the UK domestic sector , 1999 .

[103]  Chris Arney Nudge: Improving Decisions about Health, Wealth, and Happiness , 2015 .

[104]  S. Sorrell,et al.  Empirical estimates of the direct rebound effect: A review , 2009 .

[105]  Loren Lutzenhiser,et al.  A theory-based approach to market transformation , 2000 .

[106]  James J. Porter,et al.  What do we know about UK household adaptation to climate change? A systematic review , 2014, Climatic Change.

[107]  Ian Richardson,et al.  Smart meter data: Balancing consumer privacy concerns with legitimate applications , 2012 .

[108]  Donna Ciliska,et al.  Expediting systematic reviews: methods and implications of rapid reviews , 2010, Implementation science : IS.

[109]  Kathleen L. Spees,et al.  Demand Response and Electricity Market Efficiency , 2007 .

[110]  D. Bunn,et al.  Television peaks in electricity demand , 1983 .

[111]  M. Barrett,et al.  A Renewable Electricity System for the UK , 2007 .

[112]  Sophie Castle-Clarke,et al.  What are the major barriers to increased use of modern energy services among the world’s poorest people and are interventions to overcome these effective? Systematic Review. , 2011 .

[113]  P. Warren A review of demand-side management policy in the UK , 2014 .

[114]  Allan McConnell Policy Success, Policy Failure and Grey Areas In-Between , 2010 .

[115]  S. Hadley Utility DSM Programs from 1989 Through 1998: Continuation or Cross-Roads? , 1995 .

[116]  A. Harden,et al.  Methodological Issues in Combining Diverse Study Types in Systematic Reviews , 2005 .

[117]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[118]  D. Gough,et al.  An Introduction to Systematic Reviews , 2017 .

[119]  Mary Dixon-Woods,et al.  Synthesising qualitative and quantitative evidence: a review of possible methods. , 2005, Journal of health services research & policy.

[120]  Howard Geller,et al.  Utility DSM: What have we learned? Where are we going? , 1996 .

[121]  Aviel Verbruggen,et al.  Renewable and nuclear power: A common future? , 2008 .

[122]  Karsten Neuhoff Planetary Economics : Energy , Climate Change and the Three Domains of Sustainable Development , 2014 .

[123]  Jin-ho Kim,et al.  Common failures of demand response , 2011 .

[124]  Nina Zheng Evaluation of Local Enforcement of Energy Efficiency Standards and Labeling Program in China , 2013 .

[125]  J. Hills,et al.  Getting the measure of fuel poverty: final report of the Fuel Poverty Review , 2012 .

[126]  Barry D. Solomon,et al.  Eco-labeling for energy efficiency and sustainability: a meta-evaluation of US programs , 2003 .

[127]  S. Sorrell The rebound effect: an assessment of the evidence for economy-wide energy savings from improved energy efficiency , 2007 .

[128]  Paul Waide,et al.  Utility energy efficiency schemes: savings obligations and trading , 2008 .

[129]  S. Nagel Handbook of Public Policy Evaluation , 2001 .

[130]  M. Clarke,et al.  Methodology in conducting a systematic review of systematic reviews of healthcare interventions , 2011, BMC medical research methodology.

[131]  John S Rhee,et al.  Rating evidence in medical literature. , 2011, The virtual mentor : VM.

[132]  H. Madsen,et al.  Benefits and challenges of electrical demand response: A critical review , 2014 .

[133]  Bertoldi Paolo,et al.  Assessment of White Certificate Schemes and their Energy Saving Evaluation Methods , 2007 .

[134]  Franz Wirl,et al.  Storage and Demand Side Management as power generator’s strategic instruments to influence demand and prices , 2011 .

[135]  Xue Song,et al.  Historical review of demand side management in China: Management content, operation mode, results assessment and relative incentives , 2013 .

[136]  Bettina Wittneben The impact of the Fukushima nuclear accident on European energy policy , 2012 .

[137]  V. Berger Is the Jadad score the proper evaluation of trials? , 2006, The Journal of rheumatology.

[138]  J. Vandenbroucke In Defense of Case Reports and Case Series , 2001, Annals of Internal Medicine.

[139]  Patricia J. Rogers,et al.  Program theory evaluation: Practice, promise, and problems , 2000 .

[140]  Mikel González-Eguino,et al.  Handbook of Sustainable Energy , 2011 .

[141]  Vladimir Strezov,et al.  Assessment of utility energy storage options for increased renewable energy penetration , 2012 .

[142]  Edward Vine,et al.  EXAMINING CALIFORNIA'S ENERGY EFFICIENCY POLICY RESPONSE TO THE 2000/2001 ELECTRICITY CRISIS: PRACTICAL LESSONS LEARNED REGARDING POLICIES, ADMINISTRATION, AND IMPLEMENTATION , 2003 .

[143]  Sandy Oliver,et al.  Narrative approaches to systematic review and synthesis of evidence for international development policy and practice , 2012 .

[144]  Fabio Polonara,et al.  State of the art of thermal storage for demand-side management , 2012 .

[145]  Furong Li,et al.  Demand response in the UK's domestic sector , 2009 .

[146]  J. Eto,et al.  The Theory and Practice of Decoupling , 2010 .

[147]  Clark W. Gellings,et al.  Then and now: The perspective of the man who coined the term `DSM' , 1996 .

[149]  Sandy Smith,et al.  An Evaluation of the EnergyGuide Label: What we learned. , 2003 .

[150]  an Rosenowa,et al.  Evaluating the evaluations: Evidence from energy efficiency programmes in Germany and the UK , 2013 .

[151]  Tim. Chatterton,et al.  An introduction to thinking about 'energy behaviour': A multi-model approach , 2011 .

[152]  H. Dowlatabadi,et al.  Models of Decision Making and Residential Energy Use , 2007, Renewable Energy.

[153]  Mark Jaccard,et al.  The ‘neighbor effect’: Simulating dynamics in consumer preferences for new vehicle technologies , 2008 .

[154]  Benigna Boza-Kiss,et al.  Evaluating policy instruments to foster energy efficiency for the sustainable transformation of buildings , 2013 .

[155]  Yongzhen Yu,et al.  How to fit demand side management (DSM) into current Chinese electricity system reform , 2012 .

[156]  Lena Neij,et al.  A meta-analysis of bottom-up ex-ante energy policy evaluation studies , 2010 .

[157]  Howard Geller,et al.  Market Transformation Strategies to Promote End-Use Efficiency , 1994 .

[158]  Colin J. Bennett How States Utilize Foreign Evidence , 1991, Journal of Public Policy.

[159]  Interpretation and Intention in Policy Analysis , 2017 .

[160]  Ray Pawson,et al.  Evidence-based Policy: The Promise of `Realist Synthesis' , 2002 .

[161]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences , 2006 .

[162]  Liana Mirela Cipcigan,et al.  Policy and regulation for smart grids in the United Kingdom , 2014 .

[163]  Jan Rosenow,et al.  Different paths of change: home energy efficiency policy in britain and germany , 2011 .

[164]  M. Eissa Demand side management program evaluation based on industrial and commercial field data , 2011 .

[165]  M. Togeby,et al.  Energy Efficiency Obligation Schemes in the EU : Lessons Learned from Denmark , 2013 .

[166]  Peter Warren,et al.  The use of systematic reviews to analyse demand-side management policy , 2014 .

[167]  S. Sorrell,et al.  Estimating direct and indirect rebound effects for UKhouseholds , 2012 .

[168]  Patrick Devine-Wright,et al.  From demand side management to demand side participation: tracing an environmental psychology of sustainable electricity system evolution , 2004 .

[169]  Nan Zhou,et al.  Assessment of building energy-saving policies and programs in China during the 11th Five-Year Plan , 2011, Energy Efficiency.

[170]  C. Bennett,et al.  What Is Policy Convergence and What Causes It? , 1991, British Journal of Political Science.

[171]  K. Gillingham,et al.  ENERGY EFFICIENCY POLICIES: A Retrospective Examination , 2006 .

[172]  Library Services,et al.  Doing a literature search , 2011 .

[173]  R. ChittendenFandOakey,et al.  The Department of Trade and Industry , 2007 .

[174]  Tianzhen Hong,et al.  A close look at the China Design Standard for Energy Efficiency of Public Buildings , 2009 .

[175]  Brandy Brown,et al.  A Metaevaluation of Energy Efficiency Evaluations , 2014 .

[176]  David C. Ribar,et al.  Analysis of panel data, 2nd edition , 2004 .

[177]  Nebojsa Nakicenovic,et al.  Towards sustainability of energy systems: A primer on how to apply the concept of energy services to identify necessary trends and policies , 2008 .

[178]  Bert-Jaap Koops,et al.  Smart Metering and Privacy in Europe: Lessons from the Dutch Case , 2013, European Data Protection.