Secure Control under Partial Observability with Temporal Logic Constraints

This paper studies the synthesis of control policies for an agent that has to satisfy a temporal logic specification in a partially observable environment, in the presence of an adversary. The interaction of the agent (defender) with the adversary is modeled as a partially observable stochastic game. The search for policies is limited to over the space of finite state controllers, which leads to a tractable approach to determine policies. The goal is to generate a defender policy to maximize satisfaction of a given temporal logic specification under any adversary policy. We relate the satisfaction of the specification in terms of reaching (a subset of) recurrent states of a Markov chain. We then present a procedure to determine a set of defender and adversary finite state controllers of given sizes that will satisfy the temporal logic specification. We illustrate our approach with an example.

[1]  Fausto Giunchiglia,et al.  NUSMV: A New Symbolic Model Verifier , 1999, CAV.

[2]  Leslie Pack Kaelbling,et al.  Acting under uncertainty: discrete Bayesian models for mobile-robot navigation , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.

[3]  Ronen I. Brafman,et al.  A Heuristic Variable Grid Solution Method for POMDPs , 1997, AAAI/IAAI.

[4]  Krishnendu Chatterjee,et al.  A survey of partial-observation stochastic parity games , 2012, Formal Methods in System Design.

[5]  Joel W. Burdick,et al.  Finite state control of POMDPs with LTL specifications , 2014, 2014 American Control Conference.

[6]  Eric A. Hansen,et al.  Synthesis of Hierarchical Finite-State Controllers for POMDPs , 2003, ICAPS.

[7]  Calin Belta,et al.  Temporal Logic Motion Planning and Control With Probabilistic Satisfaction Guarantees , 2012, IEEE Transactions on Robotics.

[8]  Emilio Frazzoli,et al.  Control of probabilistic systems under dynamic, partially known environments with temporal logic specifications , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[9]  David Barber,et al.  On the Computational Complexity of Stochastic Controller Optimization in POMDPs , 2011, TOCT.

[10]  Hadas Kress-Gazit,et al.  Where's Waldo? Sensor-Based Temporal Logic Motion Planning , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[11]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[12]  Julia E. Sullivan,et al.  How cyber-attacks in Ukraine show the vulnerability of the U.S. power grid , 2017 .

[13]  Ufuk Topcu,et al.  Synthesis of Joint Control and Active Sensing Strategies Under Temporal Logic Constraints , 2016, IEEE Transactions on Automatic Control.

[14]  Shlomo Zilberstein,et al.  Optimizing fixed-size stochastic controllers for POMDPs and decentralized POMDPs , 2010, Autonomous Agents and Multi-Agent Systems.

[15]  Paulo Tabuada,et al.  Non-invasive Spoofing Attacks for Anti-lock Braking Systems , 2013, CHES.

[16]  Frans A. Oliehoek,et al.  A Concise Introduction to Decentralized POMDPs , 2016, SpringerBriefs in Intelligent Systems.

[17]  Krishnendu Chatterjee,et al.  The Complexity of Partial-observation Stochastic Parity Games With Finite-memory Strategies , 2014, FoSSaCS.

[18]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[19]  Christel Baier,et al.  Principles of model checking , 2008 .

[20]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[21]  Calin Belta,et al.  Formal Verification and Synthesis for Discrete-Time Stochastic Systems , 2015, IEEE Trans. Autom. Control..

[22]  François Charpillet,et al.  An Optimal Best-First Search Algorithm for Solving Infinite Horizon DEC-POMDPs , 2005, ECML.

[23]  Rafal Rohozinski,et al.  Stuxnet and the Future of Cyber War , 2011 .

[24]  Jill Slay,et al.  Lessons Learned from the Maroochy Water Breach , 2007, Critical Infrastructure Protection.

[25]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[26]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[27]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[28]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[29]  Shlomo Zilberstein,et al.  Dynamic Programming for Partially Observable Stochastic Games , 2004, AAAI.

[30]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[31]  Wayne H. Wolf,et al.  Cyber-physical Systems , 2009, Computer.

[32]  Ufuk Topcu,et al.  Receding Horizon Temporal Logic Planning , 2012, IEEE Transactions on Automatic Control.

[33]  Shlomo Zilberstein,et al.  Bounded Policy Iteration for Decentralized POMDPs , 2005, IJCAI.

[34]  Dimitri P. Bertsekas,et al.  On Near Optimality of the Set of Finite-State Controllers for Average Cost POMDP , 2008, Math. Oper. Res..

[35]  Rangoli Sharan,et al.  Formal Methods for Control Synthesis in Partially Observed Environments: Application to Autonomous Robotic Manipulation , 2014 .