Low Cost Energy Forecasting for Smart Grids Using Stream Mine 3G and Amazon EC2

As natural resources have become scarce and more costly throughout the past years, efficient resource management techniques such as smart grids have gained increasing importance. Smart grids enable an efficient power grid through the usage of information technologies. For example, demand forecasting based on energy consumption data provided by smart meters enables matching production and demand more closely. However, the processing of such smart meter data imposes several challenges: First, there is a steadily increase of data with each new installations of smart meters, with millions of devices going online each year, and, second, the complexity in the data processing tasks is characterized by heavy fluctuations during the course of a day due to behavioral patterns. Hence, the data processing technology as well as the underlying infrastructure must be (i) massively scalable and (ii) elastic, in order to cope with the huge amount of data and its daily fluctuations. Cloud computing offers a cost efficient alternative to dedicated data centers when the required amount of computational resources is fluctuating and unknown upfront. In this paper, we explore the combination of an elastic event stream processing (ESP) system named Stream Mine3G and cloud technologies such as Amazon EC2 in the context of energy forecasting. We will outline challenges with regards to scalability using two typical queries used in smart grids and demonstrate the benefits of combining elastic ESP systems with cloud technology.