Probabilistic Supervisory Control of Probabilistic Discrete Event Systems

This paper considers supervisory control of probabilistic discrete event systems (PDESs). PDESs are modeled as generators of probabilistic languages. The supervisory control problem considered is to find, if possible, a supervisor under whose control the behavior of a plant is identical to a given probabilistic specification. The probabilistic supervisors we employ are a generalization of the deterministic ones previously employed in the literature. At any state, the supervisor enables/disables events with certain probabilities. Necessary and sufficient conditions for the existence of such a supervisor, and an algorithm for its computation are presented.

[1]  Michael K. Molloy.  Performance Analysis Using Stochastic Petri Nets , 1982, IEEE Transactions on Computers.

[2]  Michael Huth,et al.  Comparing CTL and PCTL on labeled Markov chains , 1998, PROCOMET.

[3]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[4]  V. Kreinovich.  Computational Complexity and Feasibility of Data Processing and Interval Computations , 1997 .

[5]  Vijay K. Garg,et al.  Control of stochastic discrete event systems: synthesis , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[6]  R. Bellman.  Dynamic programming. , 1957, Science.

[7]  Jan J. M. M. Rutten,et al.  Mathematical techniques for analyzing concurrent and probabilistic systems , 2004, CRM monograph series.

[8]  P. Ramadge,et al.  Supervisory control of a class of discrete event processes , 1987 .

[9]  Karl Johan Åström,et al.  Optimal control of Markov processes with incomplete state information , 1965 .

[10]  Radha Jagadeesan,et al.  Metrics for Labeled Markov Systems , 1999, CONCUR.

[11]  James Worrell,et al.  An Algorithm for Quantitative Verification of Probabilistic Transition Systems , 2001, CONCUR.

[12]  Radha Jagadeesan,et al.  Metrics for labelled Markov processes , 2004, Theor. Comput. Sci..

[13]  den Ji Jerry Hartog.  Comparative semantics for a process language with probabilistic choice and non-determinism , 1998 .

[14]  Marta Z. Kwiatkowska,et al.  Stochastic Model Checking , 2007, SFM.

[15]  C. Baier,et al.  Domain equations for probabilistic processes , 2000, Mathematical Structures in Computer Science.

[16]  Dexter Kozen,et al.  A probabilistic PDL , 1983, J. Comput. Syst. Sci..

[17]  John F. Canny,et al.  Some algebraic and geometric computations in PSPACE , 1988, STOC '88.

[18]  Vijay K. Garg.  Control of Stochastic Discrete Event Systems: Existence , 1998 .

[19]  Mark Lawford,et al.  Use of a metric in supervisory control of probabilistic discrete event systems , 2010, WODES.

[20]  James Worrell,et al.  Approximating and computing behavioural distances in probabilistic transition systems , 2006, Theor. Comput. Sci..

[21]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[22]  E. Dynkin.  Controlled Random Sequences , 1965 .

[23]  Doina Precup,et al.  Methods for Computing State Similarity in Markov Decision Processes , 2006, UAI.

[24]  Edwin K. P. Chong,et al.  Discrete event systems: Modeling and performance analysis , 1994, Discret. Event Dyn. Syst..

[25]  G. Norman.  METRIC SEMANTICS FOR REACTIVE PROBABILISTIC PROCESSES , 1998 .

[26]  Kousha Etessami,et al.  Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations , 2005, JACM.

[27]  Lotfi A. Zadeh,et al.  Note on fuzzy languages , 1969, Inf. Sci..

[28]  James Worrell,et al.  Towards Quantitative Verification of Probabilistic Transition Systems , 2001, ICALP.

[29]  N. Vieille,et al.  BLACKWELL OPTIMALITY IN MARKOV DECISION PROCESSES WITH PARTIAL OBSERVATION , 2002 .

[30]  Eilon Solan,et al.  Randomization and simplification in dynamic decision-making , 2003, J. Econ. Theory.

[31]  Asok Ray,et al.  Autonomous robot navigation using optimal control of probabilistic regular languages , 2009, Int. J. Control.

[32]  James Renegar,et al.  On the Computational Complexity and Geometry of the First-Order Theory of the Reals, Part III: Quantifier Elimination , 1992, J. Symb. Comput..

[33]  Kim G. Larsen,et al.  Bisimulation through Probabilistic Testing , 1991, Inf. Comput..

[34]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .

[35]  André Arnold,et al.  Finite transition systems , 1994 .

[36]  S. E. Brodie.  New York, New York, USA , 1996 .

[37]  Alvin W Drake,et al.  Observation of a Markov process through a noisy channel , 1962 .

[38]  J. Renegar,et al.  On the Computational Complexity and Geometry of the First-Order Theory of the Reals, Part I , 1989 .

[39]  V. Borkar.  Topics in controlled Markov chains , 1991 .

[40]  Robin Milner,et al.  Algebraic laws for nondeterminism and concurrency , 1985, JACM.

[41]  Sekhar Tangirala,et al.  Controlled Markov chains with safety upper bound , 2003, IEEE Trans. Autom. Control..

[42]  Danièle Beauquier,et al.  On the Complexity of Finite Memory Policies for Markov Decision Processes , 1995, MFCS.

[43]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[44]  Bernhard Steffen,et al.  Reactive, Generative and Stratified Models of Probabilistic Processes , 1995, Inf. Comput..

[45]  Yonghua Li,et al.  Supervisory control of probabilistic discrete-event systems with recovery , 1999, IEEE Trans. Autom. Control..

[46]  Erik P. de Vink,et al.  Bisimulation for Probabilistic Transition Systems: A Coalgebraic Approach , 1999, Theor. Comput. Sci..

[47]  P. Ramadge,et al.  On the supremal controllable sublanguage of a given language , 1984, The 23rd IEEE Conference on Decision and Control.

[48]  Vijay K. Garg,et al.  An algebraic approach to modeling probabilistic discrete event systems , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[49]  Paulo Mateus,et al.  Universal aspects of probabilistic automata , 2002, Mathematical Structures in Computer Science.

[50]  Antonín Kucera,et al.  On the Controller Synthesis for Finite-State Markov Decision Processes , 2005, Fundam. Informaticae.

[51]  Doina Precup,et al.  Metrics for Markov Decision Processes with Infinite State Spaces , 2005, UAI.

[52]  Vijay K. Garg,et al.  Control of stochastic discrete event systems modeled by probabilistic languages , 2001, IEEE Trans. Autom. Control..

[53]  S. Lafortune,et al.  Using bisimulation to solve discrete event control problems , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[54]  Asok Ray,et al.  ν☆: a robot path planning algorithm based on renormalised measure of probabilistic regular languages , 2009, Int. J. Control.

[55]  Christel Baier,et al.  Controller Synthesis for Probabilistic Systems , 2004, IFIP TCS.

[56]  I. Gihman,et al.  Controlled Stochastic Processes , 1979 .

[57]  Doina Precup,et al.  Metrics for Finite Markov Decision Processes , 2004, AAAI.

[58]  W. Fleming.  Book Review: Discrete-time Markov control processes: Basic optimality criteria , 1997 .

[59]  James Renegar,et al.  On the Computational Complexity and Geometry of the First-Order Theory of the Reals, Part I: Introduction. Preliminaries. The Geometry of Semi-Algebraic Sets. The Decision Problem for the Existential Theory of the Reals , 1992, J. Symb. Comput..

[60]  Thomas A. Henzinger,et al.  Discounting the Future in Systems Theory , 2003, ICALP.

[61]  Steven Postma.  Computation of Probabilistic Supervisory Controllers for Model Matching , .

[62]  Arto Salomaa.  Formal Languages and Power Series , 1990, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[63]  Joseph F. Hanna,et al.  The structure of responses to a sequence of binary events , 1966 .

[64]  Tom Chothia,et al.  Metrics for Action-labelled Quantitative Transition Systems , 2006, QAPL.

[65]  W. M. Wonham,et al.  Supervisory control of probabilistic discrete event systems , 1993, Proceedings of 36th Midwest Symposium on Circuits and Systems.

[66]  James Worrell,et al.  An Accessible Approach to Behavioural Pseudometrics , 2005, ICALP.

[67]  Radha Jagadeesan,et al.  The metric analogue of weak bisimulation for probabilistic processes , 2002, Proceedings 17th Annual IEEE Symposium on Logic in Computer Science.

[68]  Ari Arapostathis,et al.  Control of Markov chains with safety bounds , 2005, IEEE Transactions on Automation Science and Engineering.

[69]  Azaria Paz,et al.  Probabilistic automata , 2003 .

[70]  Scott A. Smolka,et al.  Algebraic Reasoning for Probabilistic Concurrent Systems , 1990, Programming Concepts and Methods.

[71]  Alan Bundy,et al.  Constructing Induction Rules for Deductive Synthesis Proofs , 2006, CLASE.

[72]  Mark Lawford,et al.  Towards Optimal Supervisory Control of Probabilistic Discrete Event Systems , 2009 .

[73]  D. Blackwell.  Discrete Dynamic Programming , 1962 .

[74]  James Worrell,et al.  A behavioural pseudometric for probabilistic transition systems , 2005, Theor. Comput. Sci..

[75]  Scott A. Smolka,et al.  Equivalences, Congruences, and Complete Axiomatizations for Probabilistic Processes , 1990, CONCUR.

[76]  Asok Ray,et al.  Structural transformations of probabilistic finite state machines , 2008, Int. J. Control.

[77]  Asok Ray,et al.  A real time implementable All-Pair Dynamic Planning Algorithm for robot navigation based on the renormalized measure of probabilistic regular languages , 2009, 2009 American Control Conference.

[78]  Asok Ray,et al.  Optimal control of infinite horizon partially observable decision processes modelled as generators of probabilistic regular languages , 2009, Int. J. Control.

[79]  M. K. Ghosh,et al.  Discrete-time controlled Markov processes with average cost criterion: a survey , 1993 .

[80]  James Worrell,et al.  Approximating a Behavioural Pseudometric without Discount for Probabilistic Systems , 2007, Log. Methods Comput. Sci..

[81]  Eugene A. Feinberg,et al.  OPTIMALITY OF RANDOMIZED TRUNK RESERVATION FOR A PROBLEM WITH MULTIPLE CONSTRAINTS , 2007, Probability in the Engineering and Informational Sciences.

[82]  M. Reiman,et al.  Optimality of Randomized Trunk Reservation , 1994 .

[83]  Ratnesh Kumar,et al.  A probabilistic language formalism for stochastic discrete-event systems , 1999, IEEE Trans. Autom. Control..