Provision of regulation service reserves by flexible distributed loads

Following our previous work on the control of multiple appliances in response to Independent System Operator (ISO) Regulation Service Signals (RSS), we model the ISO's RSS dynamics - evolving in a time scale of seconds - as a two level Markov process whose transition probabilities are calibrated on actual data. Appliance response is modeled as a Markov modulated process consistent with an exponentially distributed time to switch off and an expected utility that is concave in the price charged when an appliance turns on. Prices are broadcasted dynamically by a Smart Building Operator (SBO) with the objective of maximizing the time average of utility gained when appliances turn on minus the cost of imperfect RSS tracking. We prove certain properties of the stochastic Dynamic Programming (DP) policies that allow us to formulate the problem as an approximate Discrete State and Control Space DP and propose a reasonable approximation that renders the problem scalable to multiple appliance categories. The discretized state DP solution can be obtained as a solution to a Linear Program (LP). The LP provides the optimal dynamic price control policies and in addition yields the requisite information needed by the SBO to bid optimally for energy and Regulation Service Reserve (RSR) Capacity in the hour ahead balancing market. Solving for the discretized real time market optimal price policies, using them to calibrate a continuous analytic policy function, and extracting from the real time optimal policies the optimal bid to the hour ahead forward balancing market is the main contribution of this paper.