Augmenting Learning Components for Safety in Resource Constrained Autonomous Robots

Learning enabled components (LECs) trained using data-driven algorithms are increasingly being used in autonomous robots commonly found in factories, hospitals, and educational laboratories. However, these LECs do not provide any safety guarantees, and testing them is challenging. In this paper, we introduce a framework that performs weighted simplex strategy based supervised safety control, resource management and confidence estimation of autonomous robots. Specifically, we describe two weighted simplex strategies: (a) simple weighted simplex strategy (SW-Simplex) that computes a weighted controller output by comparing the decisions between a safety supervisor and an LEC, and (b) a context-sensitive weighted simplex strategy (CSW-Simplex) that computes a context-aware weighted controller output. We use reinforcement learning to learn the contextual weights. We also introduce a system monitor that uses the current state information and a Bayesian network model learned from past data to estimate the probability of the robotic system staying in the safe working region. To aid resource constrained robots in performing complex computations of these weighted simplex strategies, we describe a resource manager that offloads tasks to an available fog nodes. The paper also describes a hardware testbed called DeepNNCar, which is a low cost resource-constrained RC car, built to perform autonomous driving. Using the hardware, we show that both SW-Simplex and CSW-Simplex have 40% and 60% fewer safety violations, while demonstrating higher optimized speed during indoor driving (~ 0.40 m/s) than the original system (using only LECs).

[1]  Alessio Lomuscio,et al.  An approach to reachability analysis for feed-forward ReLU neural networks , 2017, ArXiv.

[2]  Weiming Xiang,et al.  Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations , 2017, ArXiv.

[3]  Uwe Aßmann,et al.  A Capability-based Framework for Programming Small Domestic Service Robots , 2015 .

[4]  Marco Caccamo,et al.  Sandboxing Controllers for Cyber-Physical Systems , 2011, 2011 IEEE/ACM Second International Conference on Cyber-Physical Systems.

[5]  Gabor Karsai,et al.  Device Access Abstractions for Resilient Information Architecture Platform for Smart Grid , 2019, IEEE Embedded Systems Letters.

[6]  Pieter Hintjens,et al.  ZeroMQ: Messaging for Many Applications , 2013 .

[7]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[8]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Larry H. Matthies,et al.  Autonomous off‐road navigation with end‐to‐end learning for the LAGR program , 2009, J. Field Robotics.

[10]  Xin He,et al.  Attacking Vision-based Perception in End-to-End Autonomous Driving Models , 2019, J. Syst. Archit..

[11]  C. Willmott,et al.  Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance , 2005 .

[12]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[13]  Lui Sha,et al.  A Case Study on Analytical Analysis of the Inverted Pendulum Real-Time Control System , 1999 .

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  Lui Sha,et al.  Using Simplicity to Control Complexity , 2001, IEEE Softw..

[16]  Jean-Luc Dugelay,et al.  Autonomous drones for assisting rescue services within the context of natural disasters , 2014, 2014 XXXIth URSI General Assembly and Scientific Symposium (URSI GASS).

[17]  D. Jimenez,et al.  Dynamically weighted ensemble neural networks for classification , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[18]  Gul A. Agha,et al.  ACTORS - a model of concurrent computation in distributed systems , 1985, MIT Press series in artificial intelligence.

[19]  John D. Schierman,et al.  A Component-Based Simplex Architecture for High-Assurance Cyber-Physical Systems , 2017, 2017 17th International Conference on Application of Concurrency to System Design (ACSD).

[20]  Alex Fridman,et al.  Arguing Machines: Perception-Control System Redundancy and Edge Case Discovery in Real-World Autonomous Driving , 2017, ArXiv.

[21]  Zhun Fan,et al.  Service robots for hospitals: A case study of transportation tasks in a hospital , 2009, 2009 IEEE International Conference on Automation and Logistics.

[22]  Suman Jana,et al.  DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[23]  Alex Fridman,et al.  Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions , 2017, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[24]  Lui Sha,et al.  The Simplex architecture for safe online control system upgrades , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[25]  Luca Maria Gambardella,et al.  Vision-based Control of a Quadrotor in User Proximity: Mediated vs End-to-End Learning Approaches , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[26]  Yann LeCun,et al.  Off-Road Obstacle Avoidance through End-to-End Learning , 2005, NIPS.

[27]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[28]  John C. Knight,et al.  Assurance Based Development of Critical Systems , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[29]  Danbing Seto,et al.  Case Study: Development of a Baseline Controller for Automatic Landing of an F-16 Aircraft Using Linear Matrix Inequalities (LMIs) , 2000 .

[30]  Heechul Yun,et al.  DeepPicar: A Low-Cost Deep Neural Network-Based Autonomous Car , 2017, 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA).

[31]  Sertac Karaman,et al.  Project-based, collaborative, algorithmic robotics for high school students: Programming self-driving race cars at MIT , 2017, 2017 IEEE Integrated STEM Education Conference (ISEC).

[32]  Olivier Bournez,et al.  Approximate Reachability Analysis of Piecewise-Linear Dynamical Systems , 2000, HSCC.

[33]  Enhong Chen,et al.  Context-aware query classification , 2009, SIGIR.

[34]  Lui Sha,et al.  The System-Level Simplex Architecture for Improved Real-Time Embedded System Safety , 2009, 2009 15th IEEE Real-Time and Embedded Technology and Applications Symposium.

[35]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[36]  Steven Bohez,et al.  Middleware Platform for Distributed Applications Incorporating Robots, Sensors and the Cloud , 2016, 2016 5th IEEE International Conference on Cloud Networking (Cloudnet).

[37]  Insup Lee,et al.  A Study on Run Time Assurance for Complex Cyber Physical Systems , 2013 .

[38]  Gautam Biswas,et al.  An Approach To Mode and Anomaly Detection with Spacecraft Telemetry Data , 2020 .

[39]  Lui Sha,et al.  The Simplex Reference Model: Limiting Fault-Propagation Due to Unreliable Components in Cyber-Physical System Architectures , 2007, RTSS 2007.

[40]  Éric Gaussier,et al.  A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation , 2005, ECIR.

[41]  Heechul Yun,et al.  A Simplex Architecture for Intelligent and Safe Unmanned Aerial Vehicles , 2016, 2016 IEEE 22nd International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA).

[42]  Kevin McFall,et al.  Using Visual Lane Detection to Control Steering in a Self-driving Vehicle , 2016 .

[43]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[44]  Junfeng Yang,et al.  DeepXplore: Automated Whitebox Testing of Deep Learning Systems , 2017, SOSP.

[45]  Lui Sha,et al.  A dependable online testing and upgrade architecture for real-time embedded systems , 2005, 11th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA'05).

[46]  Weicun Zhang,et al.  Stable Weighted Multiple Model Adaptive Control with Improved Convergence Rate , 2012, ROCOND.

[47]  Charles Richter,et al.  Bayesian Learning for Safe High-Speed Navigation in Unknown Environments , 2015, ISRR.

[48]  Simon King,et al.  Towards context-aware face recognition , 2005, MULTIMEDIA '05.

[49]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[50]  Steven Bohez,et al.  Enabling component-based mobile cloud computing with the AIOLOS middleware , 2014, ARM '14.

[51]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[52]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[53]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[54]  Houssam Abbas,et al.  F1/10: An Open-Source Autonomous Cyber-Physical Platform , 2019, ArXiv.

[55]  S. Ullman Against direct perception , 1980, Behavioral and Brain Sciences.

[56]  Gabor Karsai,et al.  Model-based software health management for real-time systems , 2011, 2011 Aerospace Conference.

[57]  Aniruddha S. Gokhale,et al.  FECBench: An Extensible Framework for Pinpointing Sources of Performance Interference in the Cloud-Edge Resource Spectrum , 2018, 2018 IEEE/ACM Symposium on Edge Computing (SEC).

[58]  Tobias Glasmachers,et al.  Limits of End-to-End Learning , 2017, ACML.

[59]  Arvind Easwaran,et al.  CLAIR: A Contract-Based Framework for Developing Resilient CPS Architectures , 2018, 2018 IEEE 21st International Symposium on Real-Time Distributed Computing (ISORC).

[60]  Pingyu Jiang,et al.  A Cyber-physical System Architecture in Shop Floor for Intelligent Manufacturing☆ , 2016 .