$\mathtt {SIEGE}$SIEGE: A Semantics-Guided Safety Enhancement Framework for AI-Enabled C

Cyber-Physical Systems (CPSs) have been widely adopted in various industry domains to support many important tasks that impact our daily lives, such as automotive vehicles, robotics manufacturing, and energy systems. As Artificial Intelligence (AI) has demonstrated its promising abilities in diverse tasks like decision-making, prediction, and optimization, a growing number of CPSs adopt AI components in the loop to further extend their efficiency and performance. However, these modern AI-enabled CPSs have to tackle pivotal problems that the AI-enabled control systems might need to compensate the balance across multiple operation requirements and avoid possible defections in advance to safeguard human lives and properties. Modular redundancy and ensemble method are two widely adopted solutions in the traditional CPSs and AI communities to enhance the functionality and flexibility of a system. Nevertheless, there is a lack of deep understanding of the effectiveness of such ensemble design on AI-CPSs across diverse industrial applications. Considering the complexity of AI-CPSs, existing ensemble methods fall short of handling such huge state space and sophisticated system dynamics. Furthermore, an ideal control solution should consider the multiple system specifications in real-time and avoid erroneous behaviors beforehand. Such that, a new specification-oriented ensemble control system is of urgent need for AI-CPSs. In this paper, we propose $\mathtt {SIEGE}$SIEGE, a semantics-guided ensemble control framework to initiate an early exploratory study of ensemble methods on AI-CPSs and aim to construct an efficient, robust, and reliable control solution for multi-tasks AI-CPSs. We first utilize a semantic-based abstraction to decompose the large state space, capture the ongoing system status and predict future conditions in terms of the satisfaction of specifications. We propose a series of new semantics-aware ensemble strategies and an end-to-end Deep Reinforcement Learning (DRL) hierarchical ensemble method to improve the flexibility and reliability of the control systems. Our large-scale, comprehensive evaluations over five subject CPSs show that 1) the semantics abstraction can efficiently narrow the large state space and predict the semantics of incoming states, 2) our semantics-guided methods outperform state-of-the-art individual controllers and traditional ensemble methods, and 3) the DRL hierarchical ensemble approach shows promising capabilities to deliver a more robust, efficient, and safety-assured control system. To enable further research along this direction to build better AI-enabled CPS, we made all of the code and experimental results data publicly. (https://sites.google.com/view/ai-cps-siege/home).

[1]  L. Ma,et al.  FalsifAI: Falsification of AI-Enabled Hybrid Control Systems Guided by Time-Aware Coverage Criteria , 2023, IEEE Transactions on Software Engineering.

[2]  Zhenya Zhang,et al.  Online Reset for Signal Temporal Logic Monitoring , 2022, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[3]  M. Mitchell,et al.  Abstraction for Deep Reinforcement Learning , 2022, IJCAI.

[4]  Weida Wang,et al.  Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment , 2022, Energy.

[5]  L. Ma,et al.  When Cyber-Physical Systems Meet AI: A Benchmark, an Evaluation, and a Way Forward , 2021, 2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP).

[6]  Chengqing Yu,et al.  A new multi-data-driven spatiotemporal PM2.5 forecasting model based on an ensemble graph reinforcement learning convolutional network , 2021, Atmospheric Pollution Research.

[7]  Zhiwen Yu,et al.  Adaptive Subspace Optimization Ensemble Method for High-Dimensional Imbalanced Data Classification , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Angela P. Schoellig,et al.  Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning , 2021, Annu. Rev. Control. Robotics Auton. Syst..

[9]  Franco Giuseppe Dedini,et al.  Multi-objective optimization design and control of plug-in hybrid electric vehicle powertrain for minimization of energy consumption, exhaust emissions and battery degradation , 2021 .

[10]  M. A. Ganaie,et al.  Ensemble deep learning: A review , 2021, Eng. Appl. Artif. Intell..

[11]  Hui Liu,et al.  Dynamic ensemble wind speed prediction model based on hybrid deep reinforcement learning , 2021, Adv. Eng. Informatics.

[12]  S. Nanba,et al.  Ensemble Learning Method-Based Slice Admission Control for Adaptive RAN , 2020, 2020 IEEE Globecom Workshops (GC Wkshps.

[13]  Calin Belta,et al.  Recurrent Neural Network Controllers for Signal Temporal Logic Specifications Subject to Safety Constraints , 2020, IEEE Control Systems Letters.

[14]  Yang Mi,et al.  A Multi-Objective Control Strategy for Three Phase Grid-Connected Inverter During Unbalanced Voltage Sag , 2020, IEEE Transactions on Power Delivery.

[15]  Lei Ma,et al.  Marble: Model-based Robustness Analysis of Stateful Deep Learning Systems , 2020, 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[16]  Miguel G. Villarreal-Cervantes,et al.  Multi-objective meta-heuristic optimization in intelligent control: A survey on the controller tuning problem , 2020, Appl. Soft Comput..

[17]  Noorbakhsh Amiri Golilarz,et al.  A Novel Machine Learning Approach Combined with Optimization Models for Eco-efficiency Evaluation , 2020, Applied Sciences.

[18]  Hamid Parvin,et al.  Consensus function based on cluster-wise two level clustering , 2020, Artificial Intelligence Review.

[19]  P. Abbeel,et al.  SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning , 2020, ICML.

[20]  Guillermo Escrivá-Escrivá,et al.  Review on Multi-Objective Control Strategies for Distributed Generation on Inverter-Based Microgrids , 2020, Energies.

[21]  Hui Liu,et al.  A new hybrid ensemble deep reinforcement learning model for wind speed short term forecasting , 2020 .

[22]  J. P. Garrahan,et al.  A reinforcement learning approach to rare trajectory sampling , 2020, New Journal of Physics.

[23]  Mohammad Reza Keyvanpour,et al.  A new ensemble learning method based on learning automata , 2020, J. Ambient Intell. Humaniz. Comput..

[24]  Laura Wynter,et al.  A Deep Ensemble Multi-Agent Reinforcement Learning Approach for Air Traffic Control , 2020, ArXiv.

[25]  Victor Talpaert,et al.  Deep Reinforcement Learning for Autonomous Driving: A Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[26]  Zhengfei Li,et al.  Study on deep reinforcement learning techniques for building energy consumption forecasting , 2020 .

[27]  Saleh Albahli,et al.  A Deep Ensemble Learning Method for Effort-Aware Just-In-Time Defect Prediction , 2019, Future Internet.

[28]  Amjed Al-Mousa,et al.  Heart Disease Detection Using Machine Learning Majority Voting Ensemble Method , 2019, 2019 2nd International Conference on new Trends in Computing Sciences (ICTCS).

[29]  Vahid Nourani,et al.  Multi-step ahead modelling of river water quality parameters using ensemble artificial intelligence-based approach , 2019, Journal of Hydrology.

[30]  George Konidaris,et al.  On the necessity of abstraction , 2019, Current Opinion in Behavioral Sciences.

[31]  Balaraman Ravindran,et al.  SEERL: Sample Efficient Ensemble Reinforcement Learning , 2019, AAMAS.

[32]  Zhiwen Yu,et al.  A survey on ensemble learning , 2019, Frontiers of Computer Science.

[33]  Jianjun Zhao,et al.  DeepStellar: model-based quantitative analysis of stateful deep learning systems , 2019, ESEC/SIGSOFT FSE.

[34]  Torsten Braun,et al.  A Particle Filter-Based Reinforcement Learning Approach for Reliable Wireless Indoor Positioning , 2019, IEEE Journal on Selected Areas in Communications.

[35]  Taylor T. Johnson,et al.  ARCH-COMP19 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants , 2019, ARCH@CPSIoTWeek.

[36]  Florent Avellaneda,et al.  Learning Optimal Decision Trees from Large Datasets , 2019, ArXiv.

[37]  Xiaoliang Ma,et al.  A Multi-Objective Agent-Based Control Approach With Application in Intelligent Traffic Signal System , 2019, IEEE Transactions on Intelligent Transportation Systems.

[38]  M. V. Kleek,et al.  Artificial intelligence in cyber physical systems , 2019, AI & SOCIETY.

[39]  Zhiyuan Xu,et al.  Experience-Driven Congestion Control: When Multi-Path TCP Meets Deep Reinforcement Learning , 2019, IEEE Journal on Selected Areas in Communications.

[40]  Renke Huang,et al.  Adaptive Power System Emergency Control Using Deep Reinforcement Learning , 2019, IEEE Transactions on Smart Grid.

[41]  Jianli Xiao,et al.  SVM and KNN ensemble learning for traffic incident detection , 2019, Physica A: Statistical Mechanics and its Applications.

[42]  Emanuel Aldea,et al.  Evidential query-by-committee active learning for pedestrian detection in high-density crowds , 2019, Int. J. Approx. Reason..

[43]  Jehee Lee,et al.  Interactive character animation by learning multi-objective control , 2018, ACM Trans. Graph..

[44]  Robert Platt,et al.  Online abstraction with MDP homomorphisms for Deep Learning , 2018, AAMAS.

[45]  Peter Henderson,et al.  An Introduction to Deep Reinforcement Learning , 2018, Found. Trends Mach. Learn..

[46]  Daniel L. Marino,et al.  Generalization of Deep Learning for Cyber-Physical System Security: A Survey , 2018, IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society.

[47]  M. Littman,et al.  State Abstractions for Lifelong Reinforcement Learning , 2018, ICML.

[48]  Sergey Levine,et al.  Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review , 2018, ArXiv.

[49]  Lei Cao,et al.  Ensemble Network Architecture for Deep Reinforcement Learning , 2018 .

[50]  Sujata Butte,et al.  Machine Learning Based Predictive Maintenance Strategy: A Super Learning Approach with Deep Neural Networks , 2018, 2018 IEEE Workshop on Microelectronics and Electron Devices (WMED).

[51]  Herke van Hoof,et al.  Addressing Function Approximation Error in Actor-Critic Methods , 2018, ICML.

[52]  Pramod P. Khargonekar,et al.  A Sparse Neural Network Approach to Model Reference Adaptive Control with Hypersonic Flight Applications , 2018 .

[53]  Sergey Levine,et al.  Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor , 2018, ICML.

[54]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[55]  Bradley R. Schmerl,et al.  Software Engineering for Smart Cyber-Physical Systems: Challenges and Promising Solutions , 2017, SOEN.

[56]  Bernhard Rumpe,et al.  Teaching Agile Model-Driven Engineering for Cyber-Physical Systems , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET).

[57]  Pramod P. Khargonekar,et al.  Development of a robust deep recurrent neural network controller for flight applications , 2017, 2017 American Control Conference (ACC).

[58]  Dinggang Shen,et al.  Deep ensemble learning of sparse regression models for brain disease diagnosis , 2017, Medical Image Anal..

[59]  Jean Paul Barddal,et al.  A Survey on Ensemble Learning for Data Stream Classification , 2017, ACM Comput. Surv..

[60]  Jörg Holtmann,et al.  Integrated and iterative systems engineering and software requirements engineering for technical systems , 2016, J. Softw. Evol. Process..

[61]  Jameela Al-Jaroodi,et al.  Software Engineering Issues for Cyber-Physical Systems , 2016, 2016 IEEE International Conference on Smart Computing (SMARTCOMP).

[62]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[63]  Garvit Juniwal,et al.  Robust online monitoring of signal temporal logic , 2015, Formal Methods in System Design.

[64]  James Kapinski,et al.  Efficient Guiding Strategies for Testing of Temporal Properties of Hybrid Systems , 2015, NFM.

[65]  Kenneth R. Butts,et al.  Powertrain control verification benchmark , 2014, HSCC.

[66]  Cha Zhang,et al.  Ensemble Machine Learning: Methods and Applications , 2012 .

[67]  Jiafu Wan,et al.  A survey of Cyber-Physical Systems , 2011, 2011 International Conference on Wireless Communications and Signal Processing (WCSP).

[68]  Danny Hughes,et al.  Composition challenges and approaches for cyber physical systems , 2010, 2010 IEEE International Conference on Networked Embedded Systems for Enterprise Applications.

[69]  Oded Maler,et al.  Robust Satisfaction of Temporal Logic over Real-Valued Signals , 2010, FORMATS.

[70]  Wayne H. Wolf,et al.  Cyber-physical Systems , 2009, Computer.

[71]  Robert H. Gross,et al.  A novel ensemble learning method for de novo computational identification of DNA binding sites , 2007, BMC Bioinformatics.

[72]  Nadine Le Fort-Piat,et al.  Reward Function and Initial Values: Better Choices for Accelerated Goal-Directed Reinforcement Learning , 2006, ICANN.

[73]  Martin A. Riedmiller.  Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method , 2005, ECML.

[74]  Michel Verleysen,et al.  The Curse of Dimensionality in Data Mining and Time Series Prediction , 2005, IWANN.

[75]  M. Petró‐Turza,et al.  The International Organization for Standardization. , 2003 .

[76]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[77]  Robert E. Lyons,et al.  The Use of Triple-Modular Redundancy to Improve Computer Reliability , 1962, IBM J. Res. Dev..

[78]  R. Bellman.  A Markovian Decision Process , 1957 .

[79]  Haitham Abu-Rub,et al.  A novel stacked generalization ensemble-based hybrid LGBM-XGB-MLP model for Short-Term Load Forecasting , 2021, Energy.

[80]  Paolo Arcaini,et al.  Effective Hybrid System Falsification Using Monte Carlo Tree Search Guided by QB-Robustness , 2021, CAV.

[81]  P. Radanliev.  Arti � cial intelligence in cyber physical systems , 2020 .

[82]  Srinivas Koppu,et al.  Fault Control Using Triple Modular Redundancy (TMR) , 2018 .

[83]  Wenlong Fu,et al.  Model-based reinforcement learning: A survey , 2018 .

[84]  Lucian Busoniu,et al.  Reinforcement learning for control: Performance, stability, and deep approximators , 2018, Annu. Rev. Control..

[85]  Aiman H. El-Maleh,et al.  A generalized modular redundancy scheme for enhancing fault tolerance of combinational circuits , 2014, Microelectron. Reliab..

[86]  R. Buyya,et al.  Ensemble Learning , 2021, Machine Learning for Cloud Management.

[87]  Christian Engelmann,et al.  The Case for Modular Redundancy in Large-Scale High Performance Computing Systems , 2009 .

[88]  Trevor Hastie,et al.  Multi-class AdaBoost ∗ , 2009 .

[89]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[90]  Lior Rokach,et al.  Ensemble learning: A survey , 2018, WIREs Data Mining Knowl. Discov..