Effective and efficient coordination of flexibility in smart grids

Renewable energy is starting to play a serious role in the electricity world, gradually displacing the reliable (though polluting and resource-finite) conventional electricity generation technology that has served us over the last century. However, renewables offer much less control over the production of electricity, and thereby ask for new sources of flexibility. Storage is expected to become one of the key ingredients for the further development of the energy transition, as it can bridge the gap between supply and demand in time. As a lot of renewable generation is added at the lower tiers of the grid, storage can also help to keep energy local, and thereby reduce costly grid investments and transport losses, bridging the gap between supply and demand in space. Although physical energy storage (e.g. in batteries) is generally expensive, demand side management (DSM) promises to provide a different form of "storage" at (almost) no additional cost by exploiting the intrinsic flexibility within electricity consuming and producing devices. The energy transition introduces many new devices that have some flexibility in their electricity consumption or production, such as electric vehicles (EVs), heat pumps or combined heat and power (CHP) systems. What remains is to control this sea of flexibility and let the devices play their part in the smart grid. However, the control of devices in DSM turns out to be a hard problem, because the flexibility in devices is restricted, scattered, and there are costs associated with the use of the flexibility. To decide which devices are used (turned on or off) to reach some given goal, coordination is used to exploit the diversity of devices (in space). Furthermore, the control decisions impact the situation in the near future. To account for this, planning approaches may be used to exploit the flexibility of the devices over time. Together, this leads to a problem that is coupled in space and time, which is in general too large to be optimized directly, and should therefore be addressed in practice with heuristics or approximate methods. In this thesis, we address this DSM coordination/optimization problem. In this context, earlier work at the University of Twente led to TRIANA as a scalable optimization and control approach for DSM in smart grids. TRIANA partitions the optimization problem according to the hierarchical structure of the electricity grid, and splits up the DSM control problem in three phases: forecasting, planning, and real-time control. Although the approach is scalable and conceptually elegant, it simplifies the problem to such an extent that the solutions are sometimes far from being optimal. Therefore, the phases of TRIANA should be considered as dependent problems: for example, the result of real-time control depends on the forecasting and planning phases, and the planning phase should already account for this. We introduce more sophisticated planning methods (column generation and profile steering) to improve the planning results, and place these methods in a general model. To evaluate the methods, we took part in the development of an extensive simulation scenario called Flex Street. For this scenario we determine a lower bound on the cost to manage this scenario. Both of the developed planning methods bring the plan closer to the optimum than the original planning method from TRIANA (within 1-2% of the lower bound of the Flex Street scenario in a deterministic setting). A key strategy to keep the developed approaches scalable is a local optimization that already takes the needs of the nodes higher up in the hierarchical structure into account. Flexible devices are in general a major source of uncertainty themselves, since their operation depends on human behaviour, which makes the forecasting of available flexibility for specific devices difficult. Dynamic dispatch approaches address this uncertainty by exploiting the interchangeability of devices, meaning that we decide just-in-time which specific devices are going to be used, e.g. with a flexibility auction. Although this dynamic dispatching makes the approach more robust against disturbances of individual devices, it also makes the reasoning about the behaviour of the system more difficult for the planning. We propose a method to plan such a system based on the simulation of the dispatch process, where the planning result determines the configuration of a controller. We evaluate the method with a subset of Flex Street, and find that the method achieves results within 2-10% of the lower bound, depending on the considered configuration. This approach gives robust results even with large forecast errors and a small number of devices. To bring DSM methodologies to practice, there are still some barriers at a household level. One of these barriers is a limited standardization of the interface to flexible devices, leading to high software development and maintenance costs. A challenge in this standardization is that control methods differ in their perspective on flexibility. The energy flexibility interface (EFI) reacts on this challenge by proposing to communicate the structure of energy flexibility instead of a specific perspective on flexibility. We develop a comprehensive TRIANA energy application prototype that implements the EFI. The prototype supports the decentralized planning and control of real devices on low cost embedded hardware, and demonstrates that the concepts developed in this thesis are applicable in an externally given framework. It also shows that EFI maps to multiple perspectives on energy flexibility in addition to only just-in-time auction based methods. Concluding, this work lays a foundation for the further development of a flexible, effective and efficient coordination approach for flexibility in smart grids, bringing the dream of DSM - and thereby the cost effective implementation of the energy transition - a bit closer to reality.