Synthesis of plans or policies for controlling dynamic systems

To be properly controlled, dynamic systems need plans or policies. Plans are sequences of actions to be performed, whereas policies associate an action to be performed with each possible system state. The model-based synthesis of plans or policies consists in producing them automatically starting from a model of the physical system to be controlled and from user requirements on the controlled system. This article is a survey of what exists and what has been done at ONERA for the automatic synthesis of plans or policies for the high-level control of dynamic systems.

[1]  Weihong Zhang,et al.  Fast Value Iteration for Goal-Directed Markov Decision Processes , 1997, UAI.

[2]  Milos Hauskrecht,et al.  Solving Factored MDPs with Continuous and Discrete Variables , 2004, UAI.

[3]  Gérard Verfaillie,et al.  Selecting and scheduling observations of agile satellites , 2002 .

[4]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[5]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[6]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[7]  Romain Grasset-Bourdel.  Planning and replanning for a constellation of agile Earth observation satellites , 2011 .

[8]  James A. Hendler,et al.  UMCP: A Sound and Complete Procedure for Hierarchical Task-network Planning , 1994, AIPS.

[9]  Enrico Macii,et al.  Algebraic decision diagrams and their applications , 1993, Proceedings of 1993 International Conference on Computer Aided Design (ICCAD).

[10]  Marc Schoenauer,et al.  An Evolutionary Metaheuristic Based on State Decomposition for Domain-Independent Satisficing Planning , 2010, ICAPS.

[11]  E. Lawler,et al.  Erratum: The Traveling Salesman Problem: A Guided Tour of Combinatorial Optimization , 1986 .

[12]  Charles Lesire,et al.  A generic framework for anytime execution-driven planning in robotics , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Cédric Pralet,et al.  Decision upon observations and data downloads by an autonomous Earth surveillance satellite , 2007 .

[14]  Eugene L. Lawler,et al.  A Guided Tour of Combinatorial Optimization , 1985 .

[15]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[16]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[17]  Peter van Beek,et al.  CPlan: A Constraint Programming Approach to Planning , 1999, AAAI/IAAI.

[18]  John K. Slaney,et al.  Decision-Theoretic Planning with non-Markovian Rewards , 2011, J. Artif. Intell. Res..

[19]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[20]  Shlomo Zilberstein,et al.  Using Anytime Algorithms in Intelligent Systems , 1996, AI Mag..

[21]  Ivan Serina,et al.  Planning Through Stochastic Local Search and Temporal Action Graphs in LPG , 2003, J. Artif. Intell. Res..

[22]  Piergiorgio Bertoli,et al.  Planning in Nondeterministic Domains under Partial Observability via Symbolic Model Checking , 2001, IJCAI.

[23]  Charles Lesire.  Iterative Planning of Airport Ground Movements , 2010 .

[24]  Christos G. Cassandras,et al.  Introduction to Discrete Event Systems , 1999, The Kluwer International Series on Discrete Event Dynamic Systems.

[25]  Thomas Schiex,et al.  Valued Constraint Satisfaction Problems: Hard and Easy Problems , 1995, IJCAI.

[26]  Ugur Kuter,et al.  Incremental plan aggregation for generating policies in MDPs , 2010, AAMAS.

[27]  Romain Grasset-Bourdel.  Building a really executable plan for a constellation of agile Earth observation satellites , 2011 .

[28]  Richard E. Korf,et al.  Real-Time Heuristic Search , 1990, Artif. Intell..

[29]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[30]  Hector Geffner,et al.  Branching and pruning: An optimal temporal POCL planner based on constraint programming , 2004, Artif. Intell..

[31]  Maria Fox,et al.  PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains , 2003, J. Artif. Intell. Res..

[32]  Jean-Loup Farges,et al.  POMDP solving: what rewards do you really expect at execution? , 2010, STAIRS.

[33]  Bernhard Nebel,et al.  The FF Planning System: Fast Plan Generation Through Heuristic Search , 2011, J. Artif. Intell. Res..

[34]  Gerard. Verfaillie,et al.  How to Model Planning and Scheduling Problems using Timelines , 2008 .

[35]  É. Belin,et al.  Interaction between reactive and deliberative tasks for on-line decision-making , 2007 .

[36]  Sofia Cassel,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 2012 .

[37]  G. Verfaillie,et al.  Decision-Making on-board an Autonomous Agile Earth-Observing Satellite , 2008 .

[38]  Marco Pistore,et al.  Weak, strong, and strong cyclic planning via symbolic model checking , 2003, Artif. Intell..

[39]  R. Bellman.  Dynamic programming. , 1957, Science.

[40]  Paolo Traverso,et al.  Automated planning - theory and practice , 2004 .

[41]  Cédric Pralet,et al.  Constraint Programming for Controller Synthesis , 2011, CP.

[42]  Makoto Yokoo,et al.  The Distributed Constraint Satisfaction Problem: Formalization and Algorithms , 1998, IEEE Trans. Knowl. Data Eng..

[43]  Benjamin C. Kuo,et al.  AUTOMATIC CONTROL SYSTEMS , 1962, Universum:Technical sciences.

[44]  Cédric Pralet,et al.  Constraint-Based Controller Synthesis in Non-Deterministic and Partially Observable Domains , 2010, ECAI.

[45]  Philippe Baptiste,et al.  Constraint - based scheduling : applying constraint programming to scheduling problems , 2001 .

[46]  W. M. Wonham,et al.  The control of discrete event systems , 1989 .

[47]  Cédric Pralet,et al.  Using Constraint Networks on Timelines to Model and Solve Planning and Scheduling Problems , 2008, ICAPS.

[48]  John L. Bresina,et al.  Heuristic-Biased Stochastic Sampling , 1996, AAAI/IAAI, Vol. 1.

[49]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[50]  Manfred Morari,et al.  Model predictive control: Theory and practice , 1988 .

[51]  Cédric Pralet,et al.  Forward Constraint-Based Algorithms for Anytime Planning , 2009, ICAPS.

[52]  Kevin B. Korb,et al.  Bayesian Artificial Intelligence , 2004, Computer science and data analysis series.

[53]  A. TUSTIN,et al.  Automatic Control Systems , 1950, Nature.

[54]  Deeparnab Chakrabarty,et al.  Knapsack Problems , 2008 .

[55]  R. I. Bahar,et al.  Algebraic decision diagrams and their applications , 1993, Proceedings of 1993 International Conference on Computer Aided Design (ICCAD).

[56]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[57]  Shlomo Zilberstein,et al.  LAO*: A heuristic search algorithm that finds solutions with loops , 2001, Artif. Intell..

[58]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[59]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[60]  E. Allen Emerson,et al.  Temporal and Modal Logic , 1991, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[61]  Frédéric Dehais,et al.  Authority Management and Conflict Solving in Human-Machine Systems , 2012 .

[62]  Vincent Vidal,et al.  A Lookahead Strategy for Heuristic Search Planning , 2004, ICAPS.

[63]  Blai Bonet,et al.  Planning as heuristic search , 2001, Artif. Intell..

[64]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[65]  Manfred Morari,et al.  Model predictive control: Theory and practice - A survey , 1989, Autom..

[66]  Stanley J. Rosenschein,et al.  Learning to act using real-time dynamic programming , 1996 .

[67]  Bart Selman,et al.  Planning as Satisfiability , 1992, ECAI.

[68]  Guillaume Infantes,et al.  Extending Classical Planning Heuristics to Probabilistic Planning with Dead-Ends , 2011, AAAI.

[69]  Patrik Haslum,et al.  Deterministic planning in the fifth international planning competition: PDDL3 and experimental evaluation of the planners , 2009, Artif. Intell..