Lazy reinforcement learning for real-time generation control of parallel cyber-physical-social energy systems

Abstract To learn human intelligence, the social system/human system is added to a cyber–physical energy system in this paper. To accelerate the configuration process of the parameters of the cyber–physical energy system, parallel systems based on artificial societies-computational experiments-parallel execution are added to the cyber–physical energy system, i.e., a parallel cyber–physical–social energy system is proposed in this paper. This paper proposes a real-time generation control framework to replace the conventional generation control framework with multiple time scales, which consist of long-term time scale, short-term time scale, and real-time scale. Since a lazy operator employed into reinforcement learning, a lazy reinforcement learning is proposed for the real-time generation control framework. To reduce the real simulation time, multiple virtual parallel cyber–physical–social energy systems and a real parallel cyber–physical–social energy system are built for the real-time generation control of large-scale multi-area interconnected power systems. Compared with a total of 146016 conventional generation control algorithms and a relaxed artificial neural network in the simulation of IEEE 10-generator 39-bus New-England power system, the proposed lazy reinforcement learning based real-time generation control controller can obtain the highest control performance. The active power between two areas and the systemic frequency deviation can be reduced by the lazy reinforcement learning, and the simulation results verify the effectiveness and feasibility of the proposed lazy reinforcement learning based real-time generation control controller for the parallel cyber–physical–social energy systems.