Dissipativity-Based Verification for Autonomous Systems in Adversarial Environments

In this chapter, we investigate the problem of verifying desired properties of large-scale cyber-physical systems that operate in uncertain, adversarial environments. We focus on properties relating to dissipativity and we propose approaches to achieve verification with minimum knowledge of the system dynamics. In order to achieve compositionality in large-scale models, we distribute the verification process among individual subsystems, which utilize limited information received locally from their immediate neighbors. The need for knowledge of the subsystem parameters is avoided via a novel reinforcement learning-inspired approach that enables decentralized evaluation of appropriate storage functions that can be used to verify dissipativity. Our method allows the addition of learning-enabled subsystems to the cyber-physical network while dissipativity is ensured. We show how different learning rules can be used to guarantee different properties, such as \(L_2\)-gain stability or losslessness. Finally, the need for verification of complex properties is addressed and we propose new directions for this problem.

[1]  Panos J. Antsaklis,et al.  Sequential Synthesis of Distributed Controllers for Cascade Interconnected Systems , 2019, 2019 American Control Conference (ACC).

[2]  Heejo Lee,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. INVITED PAPER Cyber–Physical Security of a Smart Grid Infrastructure , 2022 .

[3]  Insup Lee,et al.  Medical Cyber Physical Systems , 2010, Design Automation Conference.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Kyriakos G. Vamvoudakis,et al.  Stochastic zero-sum nash games for uncertain nonlinear Markovian jump systems , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[6]  Frank L. Lewis,et al.  Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control , 2007, Autom..

[7]  Panos J. Antsaklis,et al.  Distributed Synthesis of Local Controllers for Networked Systems With Arbitrary Interconnection Topologies , 2019, IEEE Transactions on Automatic Control.

[8]  Frank L. Lewis,et al.  Autonomy and machine intelligence in complex systems: A tutorial , 2015, 2015 American Control Conference (ACC).

[9]  Frank L. Lewis,et al.  Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2010, Autom..

[10]  S. Shankar Sastry,et al.  Secure Control: Towards Survivable Cyber-Physical Systems , 2008, 2008 The 28th International Conference on Distributed Computing Systems Workshops.

[11]  Insup Lee,et al.  Cyber-physical systems: The next computing revolution , 2010, Design Automation Conference.

[12]  Sriram Sankaranarayanan,et al.  Probabilistic Temporal Logic Falsification of Cyber-Physical Systems , 2013, TECS.

[13]  Wenji Mao,et al.  Cyber-Physical-Social Systems for Command and Control , 2011, IEEE Intelligent Systems.

[14]  Frank L. Lewis,et al.  Optimal Control: Lewis/Optimal Control 3e , 2012 .

[15]  Petros A. Ioannou,et al.  Robust Adaptive Control , 2012 .

[16]  Ragunathan Rajkumar,et al.  Parallel scheduling for cyber-physical systems: Analysis and case study on a self-driving car , 2013, 2013 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS).

[17]  Kyriakos G. Vamvoudakis,et al.  Non-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems , 2015, Autom..

[18]  Mathukumalli Vidyasagar,et al.  New passivity-type criteria for large-scale interconnected systems , 1979 .

[19]  Etika Agarwal Compositional Control of Large-Scale Cyber-Physical Systems Using Hybrid Models and Dissipativity Theory , 2019 .

[20]  Vijay Gupta,et al.  Compositional Verification of Passivity for Cascade Interconnected Nonlinear Systems , 2020, 2020 28th Mediterranean Conference on Control and Automation (MED).

[21]  Kyriakos G. Vamvoudakis,et al.  Decentralized Verification for Dissipativity of Cascade Interconnected Systems , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[22]  Frank L. Lewis,et al.  Nonlinear H2/H-Infinity Constrained Feedback Control: A Practical Design Approach Using Neural Networks (Advances in Industrial Control) , 2006 .

[23]  Frank L. Lewis,et al.  Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems , 2014, Autom..

[24]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[25]  Frank L. Lewis,et al.  Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics , 2014, Autom..

[26]  Kyriakos G. Vamvoudakis,et al.  Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach , 2017, Syst. Control. Lett..

[27]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[28]  Panos J. Antsaklis,et al.  Large-Scale Dissipative and Passive Control Systems and the Role of Star and Cyclic Symmetries , 2016, IEEE Transactions on Automatic Control.