Scalable Clustering of Individual Electrical Curves for Profiling and Bottom-up Forecasting

Smart grids require flexible data driven forecasting methods. We propose clustering tools for bottom-up short-term load forecasting. We focus on individual consumption data analysis which plays a major role for energy management and electricity load forecasting. The two first sections are dedicated to the industrial context and a review of individual electrical data analysis. We are interested in hierarchical time-series for bottom-up forecasting. The idea is to disaggregate the signal in such a way that the sum of disaggregated forecasts improves the direct prediction. The 3-steps strategy defines numerous super-consumers by curve clustering, builds a hierarchy of partitions and selects the best one minimizing a forecast criterion. Using a nonparametric model to handle forecasting, and wavelets to define various notions of similarity between load curves, this disaggregation strategy applied to French individual consumers leads to a gain of 16% in forecast accuracy. We then explore the upscaling capacity of this strategy facing massive data and implement proposals using R, the free software environment for statistical computing. The proposed solutions to make the algorithm scalable combines data storage, parallel computing and double clustering step to define the super-consumers.