A A Tutorial on Event-Based Optimization with Application in Energy Internet

In many practical systems, the control or decision-making is usually triggered by certain events. These systems are classified as discrete event dynamic systems (DEDSs). Considering the performance optimization of these systems, a new optimization framework called event-based optimization (EBO) is introduced in this paper. Compared with Markov decision process (MDP), one of the main characteristics of EBO is that decisions are made based on“events”rather than states. In this regard, there exist a number of advantages for EBO. First, an event usually corresponds to a set of state transitions with some common properties. Generally, the number of events of a system requiring decisions is much smaller than that of states. Therefore, the EBO approach can utilize the special structure of systems characterized by events to aggregate the potentials, thus alleviating the curses of dimensionality. Second, the EBO approach applies to many practical problems where actions are required only when certain events happen. Such problems do not fit well the standard MDP formulation in which the decisions made based on different states are independent. However, for that cases, the same action may be taken for the same event, which may correspond to many different states. Based on the basic theory of MDP, this paper is addressed around three aspects. First, we briefly review the basic ideas of EBO and the development for its theory and applications. Second, we introduce the simulation-based policy iteration methods for EBO based on the performance potentials or Q-factors; Third, a case study is conducted on the coordination of electric vehicle charging with the distributed wind power generation of a building, which aims to shed some lights on the application of EBO in energy Internet.

[1]  E. J. Sondik,et al.  The Optimal Control of Partially Observable Markov Decision Processes. , 1971 .

[2]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[3]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[4]  Karl-Erik Årzén,et al.  A simple event-based PID controller , 1999 .

[5]  K. Åström,et al.  Comparison of Periodic and Event Based Sampling for First-Order Stochastic Systems , 1999 .

[6]  Liyi Dai,et al.  Introduction to discrete event systems [Book Review] , 2001, IEEE Transactions on Automatic Control.

[7]  B. Krogh,et al.  State aggregation in Markov decision processes , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[8]  Wei Wu,et al.  Optimal control of stochastic systems with costly observations - the general Markovian model and the LQG problem , 2005, Proceedings of the 2005, American Control Conference, 2005..

[9]  Xi-Ren Cao,et al.  Basic Ideas for Event-Based Optimization of Markov Systems , 2005, Discret. Event Dyn. Syst..

[10]  T. Başar,et al.  Optimal Estimation with Limited Measurements , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[11]  Dimitri P. Bertsekas,et al.  Neuro-Dynamic Programming: An Overview and Recent Results , 2006, OR.

[12]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[13]  J.P. Hespanha,et al.  A Constant Factor Approximation Algorithm for Event-Based Sampling , 2007, 2007 American Control Conference.

[14]  W. P. M. H. Heemels,et al.  Analysis of event-driven controllers for linear systems , 2008, Int. J. Control.

[15]  Li Xia,et al.  A Structure Property of Optimal Policies for Maintenance Problems WithSafety-Critical Components , 2008, IEEE Transactions on Automation Science and Engineering.

[16]  Qing-Shan Jia,et al.  Event-based optimization for dispatching policies in material handling systems of general assembly lines , 2008, 2008 47th IEEE Conference on Decision and Control.

[17]  L.H. Tsoukalas,et al.  From smart grids to an energy internet: Assumptions, architectures and requirements , 2008, 2008 Third International Conference on Electric Utility Deregulation and Restructuring and Power Technologies.

[18]  Hongyi Li,et al.  Event-based predictive control strategy for teleoperation via Internet , 2008, 2008 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[19]  Xi-Ren Cao,et al.  Stochastic learning and optimization - A sensitivity-based approach , 2007, Annual Reviews in Control.

[20]  Zhao Chuanhui,et al.  Energy Internet system based on distributed renewable energy generation , 2010 .

[21]  Qing-Shan Jia A Structural Property of Optimal Policies for Multi-Component Maintenance Problems , 2010, IEEE Transactions on Automation Science and Engineering.

[22]  Yin Baoqun Online adaptive optimization for event-driven dynamic service composition , 2011 .

[23]  A Q Huang,et al.  The Future Renewable Electric Energy Delivery and Management (FREEDM) System: The Energy Internet , 2011, Proceedings of the IEEE.

[24]  Qing-Shan Jia,et al.  On State Aggregation to Approximate Complex Value Functions in Large-Scale Markov Decision Processes , 2011, IEEE Transactions on Automatic Control.

[25]  Xi-Ren Cao,et al.  Event-Based Optimization for POMDPs and Its Application in Portfolio Management , 2011 .

[26]  Xiaofeng Wang,et al.  Event-Triggering in Distributed Networked Control Systems , 2011, IEEE Transactions on Automatic Control.

[27]  Qing-Shan Jia,et al.  On Solving Event-Based Optimization With Average Reward Over Infinite Stages , 2011, IEEE Transactions on Automatic Control.

[28]  Qing-Shan Jia,et al.  On Solving Optimal Policies for Finite-Stage Event-Based Optimization , 2011, IEEE Transactions on Automatic Control.

[29]  Li Xia,et al.  Event-based sensor activation for indoor occupant distribution estimation , 2012, 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV).

[30]  Qing-Shan Jia,et al.  Event-based evacuation in outdoor environment , 2012, CCDC 2012.

[31]  Takeshi Shibuya,et al.  Partially Observable Markov Decision Process and Reinforcement Learning , 2013 .

[32]  Frank L. Lewis,et al.  An Introduction to Event-Based Optimization: Theory and Applications , 2013 .

[33]  Ramachandra Kota,et al.  An Agent-Based Approach to Virtual Power Plants of Wind Power Generators and Electric Vehicles , 2013, IEEE Transactions on Smart Grid.

[34]  Li Xia,et al.  Event-based optimization of admission control in open queueing networks , 2014, Discret. Event Dyn. Syst..

[35]  Ruzhu Wang,et al.  Advances in wind energy resource exploitation in urban environment: A review , 2014 .

[36]  Li Xia,et al.  A tutorial on event-based optimization—a new optimization framework , 2013, Discrete Event Dynamic Systems.

[37]  Li Xia,et al.  EV charging load scheduling following uncertain renewable energy supply by stochastic matching , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[38]  Qing-Shan Jia,et al.  Matching EV Charging Load With Uncertain Wind: A Simulation-Based Policy Improvement Approach , 2015, IEEE Transactions on Smart Grid.

[39]  Peter B. Luh,et al.  Event-Based Optimization Within the Lagrangian Relaxation Framework for Energy Savings in HVAC Systems , 2015, IEEE Transactions on Automation Science and Engineering.

[40]  Junyu Zhang,et al.  A special case of partially observable Markov decision processes problem by event-based optimization , 2016, 2016 IEEE International Conference on Industrial Technology (ICIT).

[41]  Qing-Shan Jia,et al.  The joint scheduling of EV charging load with building mounted wind power using simulation-based policy improvement , 2016, 2016 International Symposium on Flexible Automation (ISFA).

[42]  Qing-Shan Jia,et al.  Optimal Control of Multiroom HVAC System: An Event-Based Approach , 2016, IEEE Transactions on Control Systems Technology.

[43]  Nikos D. Hatziargyriou,et al.  Distributed Coordination of Electric Vehicles Providing V2G Services , 2016, IEEE Transactions on Power Systems.

[44]  Adriana Chis,et al.  Reinforcement Learning-Based Plug-in Electric Vehicle Charging With Forecasted Price , 2017, IEEE Transactions on Vehicular Technology.