Optimal Control Strategy for Plug-in Electric Vehicles Based on Reinforcement Learning in Distribution Networks

Electric vehicles (EVs) as distributed storage devices have the potential to provide frequency regulation services due to the fast adjustment of charging/discharging power. Along with the policy incentives, it is practical for EVs to take part in the regulation market through the aggregator. An optimal control strategy based on reinforcement learning (RL) for electric vehicles (EVs) in distributed networks is proposed in this paper. The overall goal is to follow the regulation signals sent by the system operator in the real time regulation market by controlling the EVs in the parking lot. To achieve this, the reinforcement learning algorithm is employed to optimize the charge and discharge strategy of the EVs, so that the aggregator optimally allocates the regulation power and the baseline charging power to EVs to respond to the regulation signals for the best regulation performance. Comprehensive simulation studies have been carried out based on the data of PJM electricity market and the results show that the regulation performance based on the control strategy is excellent in both cases of traditional and dynamic regulation signals.

[1]  Zechun Hu,et al.  Decentralized Vehicle-to-Grid Control for Primary Frequency Regulation Considering Charging Demands , 2013, IEEE Transactions on Power Systems.

[2]  Massimo Santarelli,et al.  Inverse parameter determination in the development of an optimized lithium iron phosphate – Graphite battery discharge model , 2016 .

[3]  Peter Palensky,et al.  Modeling Intelligent Energy Systems: Co-Simulation Platform for Validating Flexible-Demand EV Charging Management , 2013, IEEE Transactions on Smart Grid.

[4]  Anastasios G. Bakirtzis,et al.  Optimal Bidding Strategy for Electric Vehicle Aggregators in Electricity Markets , 2013, IEEE Transactions on Power Systems.

[5]  Om Prakash Mahela,et al.  Power quality improvement in distribution network using DSTATCOM with battery energy storage system , 2016 .

[6]  Manoj Datta,et al.  Frequency Control Method using Automated Demand Response for Isolated Power System with Renewable Energy Sources , 2017 .

[7]  Sekyung Han,et al.  Performance-Based Settlement of Frequency Regulation for Electric Vehicle Aggregators , 2018, IEEE Transactions on Smart Grid.

[8]  Song Guo,et al.  Energy Management for EV Charging in Software-Defined Green Vehicle-to-Grid Network , 2018, IEEE Communications Magazine.

[9]  Min Dong,et al.  Real-Time Welfare-Maximizing Regulation Allocation in Dynamic Aggregator-EVs System , 2014, IEEE Transactions on Smart Grid.

[10]  Mushfiqur R. Sarker,et al.  Optimal participation of an electric vehicle aggregator in day-ahead energy and reserve markets , 2016 .

[11]  Anastasios G. Bakirtzis,et al.  Real-Time Charging Management Framework for Electric Vehicle Aggregators in a Market Environment , 2016, IEEE Transactions on Smart Grid.

[12]  Sanjoy Debbarma,et al.  Frequency Regulation in Deregulated Market Using Vehicle-to-Grid Services in Residential Distribution Network , 2018, IEEE Systems Journal.

[13]  Mun-Kyeom Kim,et al.  Hybrid operation strategy of wind energy storage system for power grid frequency regulation , 2016 .

[14]  Ricardo J. Bessa,et al.  Improving Renewable Energy Forecasting With a Grid of Numerical Weather Predictions , 2017, IEEE Transactions on Sustainable Energy.

[15]  Mohamed A. El-Sharkawi,et al.  Optimal Charging Strategies for Unidirectional Vehicle-to-Grid , 2011, IEEE Transactions on Smart Grid.

[16]  Zhao Yang Dong,et al.  Modeling and Analysis of Lithium Battery Operations in Spot and Frequency Regulation Service Markets in Australia Electricity Market , 2017, IEEE Transactions on Industrial Informatics.

[17]  Yang Shi,et al.  Model Predictive Control of Aggregated Heterogeneous Second-Order Thermostatically Controlled Loads for Ancillary Services , 2016, IEEE Transactions on Power Systems.

[18]  Gonzalo Seco-Granados,et al.  Fair Design of Plug-in Electric Vehicles Aggregator for V2G Regulation , 2012, IEEE Transactions on Vehicular Technology.

[19]  Chris Phillips,et al.  Cooperative collision avoidance for multi-vehicle systems using reinforcement learning , 2013, 2013 18th International Conference on Methods & Models in Automation & Robotics (MMAR).