Model Embedded Control: A Method to Rapidly Synthesize Control- lers in a Modeling Environment

One of the challenges in modeling complex systems is the creation of quality controllers. In some projects, the effort to develop even a reasonable prototype controller dwarfs the effort required to develop a physical model. For a limited class of problems, it is possible and tractable to directly synthesize a controller from a mathematical statement of control objectives and a model of the plant. To do this, a system model is decomposed into a controls model and a plant model. The controls model is further decomposed into an optimization problem and a ‘zero-time’ plant model. The zero-time plant model in the controller is a copy or a reasonable representation of the real plant model. It is used to evaluate the future impact of possible control actions. This type of controller is referred to as a Model Embedded Controller (MEC) and can be used to realize controllers designed using Dynamic Programming (DP). To illustrate this approach, an approximation to the problem of starting an engine is considered. In this problem, an electric machine with a flywheel is connected to crank and slider with a spring attached to the slider. The machine torque is constrained to a value which is insufficient to statically overcome the force of the spring. This constraint prevents the motor from achieving the desired speed from some initial conditions if it only supplies maximal torque in the desired direction of rotation. By using DP, a control strategy that achieves the desired speed from any initial condition is generated. This controller is realized in the model using MEC. The controller for this example is created by forming an optimization problem and calling an embedded copy of the plant model. Furthermore, this controller is calibrated by conducting a large scale Design of Experiments (DOE). The experiments are processed to generate the calibrations for the controller such that it achieves its design objectives when used for closed loop control of the plant model. It is well understood that Modelica includes many language features that allow plant models to be developed quickly. As discussed previously, the development of quality control strategies generally remains a bottleneck. In this paper we show how existing features along with appropriate tool support and potential language changes can make a significant impact on the model development process by supporting an automated control synthesis process.

[1]  Ilya Kolmanovsky,et al.  Approximate dynamic programming solutions for lean burn engine aftertreatment , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[2]  V. Farias Tetris : Experiments with the LP Approach to Approximate DP , 2003 .

[3]  Huei Peng,et al.  A stochastic control strategy for hybrid electric vehicles , 2004, Proceedings of the 2004 American Control Conference.

[4]  Ilya Kolmanovsky,et al.  Optimization of powertrain operating policy for feasibility assessment and calibration: stochastic dynamic programming approach , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[5]  A. Galip Ulsoy,et al.  Nested plant/controller optimization with application to combined passive/active automotive suspensions , 2003, Proceedings of the 2003 American Control Conference, 2003..

[6]  Michael James Sasena,et al.  Flexibility and efficiency enhancements for constrained global design optimization with kriging approximations. , 2002 .

[7]  John N. Tsitsiklis,et al.  Stable LInear Approximations to Dynamic Programming for Stochastic Control Problems with Local Transitions , 1995, NIPS.

[8]  Michael A. Trick,et al.  A Linear Programming Approach to Solving Stochastic Dynamic Programming , 1993 .

[9]  Geoffrey J. Gordon Stable Function Approximation in Dynamic Programming , 1995, ICML.

[10]  John Rust Using Randomization to Break the Curse of Dimensionality , 1997 .

[11]  Benjamin Van Roy,et al.  The linear programming approach to approximate dynamic programming: theory and application , 2002 .

[12]  Benjamin Van Roy,et al.  The Linear Programming Approach to Approximate Dynamic Programming , 2003, Oper. Res..

[13]  Hosam K. Fathy,et al.  Combined plant and control optimization: Theory, strategies and applications. , 2003 .

[14]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[15]  Stanley E. Zin,et al.  SPLINE APPROXIMATIONS TO VALUE FUNCTIONS: Linear Programming Approach , 1997 .

[16]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[17]  Huei Peng,et al.  Power management strategy for a parallel hybrid electric truck , 2003, IEEE Trans. Control. Syst. Technol..

[18]  Huei Peng,et al.  Shortest path stochastic control for hybrid electric vehicles , 2008 .

[19]  Philip E. Gill,et al.  Practical optimization , 1981 .

[20]  Jong Min Lee,et al.  Approximate Dynamic Programming Strategies and Their Applicability for Process Control: A Review and Future Directions , 2004 .

[21]  A. Michael,et al.  A Linear Programming Approach toSolving Stochastic Dynamic Programs , 1993 .

[22]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[24]  Andrew W. Moore,et al.  Barycentric Interpolators for Continuous Space and Time Reinforcement Learning , 1998, NIPS.

[25]  Daniela Pucci de Farias,et al.  Approximate value iteration with randomized policies , 2000, CDC 2000.

[26]  Daniela Pucci de Farias,et al.  Approximate value iteration and temporal-difference learning , 2000 .

[27]  A. Galip Ulsoy,et al.  On the coupling between the plant and controller optimization problems , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[28]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[29]  Christine A. Shoemaker,et al.  Applying Experimental Design and Regression Splines to High-Dimensional Continuous-State Stochastic Dynamic Programming , 1999, Oper. Res..

[30]  Andrew W. Moore,et al.  Variable Resolution Discretization in Optimal Control , 2002, Machine Learning.

[31]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[32]  Shie Mannor,et al.  Automatic basis function construction for approximate dynamic programming and reinforcement learning , 2006, ICML.