A unified framework for big data acquisition, storage, and analytics for demand response management in smart cities

Abstract With an increased usage of information and communication technologies (ICT) in the smart cities, the data generated from different smart devices has increased manifolds. This data is heterogeneous in nature which varies with respect to time and exhibits the properties of all essential V’s for big data. Therefore, to handle such an enormous amount of data, various big data processing techniques are required. To cope up with these issues, this paper presents a tensor-based big data management technique to reduce the dimensionality of data gathered from the Internet-of-Energy (IoE) environment in a smart city. The core data is extracted out of the gathered data by using tensor operations such as-matricization, vectorization and tensorization with the help of higher-order singular value decomposition. This core data is then stored on the cloud in the reduced form. After reducing the dimensionality of data, it is used for providing many services in smart cities; and its application to provide demand response (DR) services has been discussed in this paper. For this purpose, support vector machine (SVM)-based classifier is used to classify the end-users (residential and commercial) into normal, overloaded and underloaded categories from the core data. Once such users are identified to take part in DR mechanism, utilities then generate commands to handle their DR in order to alter load requirements so that the overall load is optimized. Results obtained on Open Energy Information and PJM dataset clearly indicate the supremacy of the proposed tensor-based scheme over the traditional scheme for DR management.