Considerations in Assuring Safety of Increasingly Autonomous Systems [STUB]

[1]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[2]  Bertrand Meyer,et al.  On Formalism in Specifications , 1985, IEEE Software.

[3]  Ufuk Topcu,et al.  Safe Reinforcement Learning via Shielding , 2017, AAAI.

[4]  Pieter Abbeel,et al.  Safe Exploration in Markov Decision Processes , 2012, ICML.

[5]  Raja Parasuraman,et al.  Human Versus Automation in Responding to Failures: An Expected-Value Analysis , 2000, Hum. Factors.

[6]  Ken Funk,et al.  Studies of Cockpit Task Management Errors , 1996 .

[7]  Shalini Ghosh,et al.  Trusted Machine Learning for Probabilistic Models , 2016 .

[8]  R. W. Witty,et al.  Safe programming , 1978 .

[9]  Linda J. Sorensen,et al.  Cognitive Work Analysis: Coping with Complexity , 2010 .

[10]  Bev Littlewood,et al.  Reasoning about the Reliability of Diverse Two-Channel Systems in Which One Channel Is "Possibly Perfect" , 2012, IEEE Transactions on Software Engineering.

[11]  V.A. Carreno,et al.  A case-study application of RTCA DO-254: design assurance guidance for airborne electronic hardware , 2000, 19th DASC. 19th Digital Avionics Systems Conference. Proceedings (Cat. No.00CH37126).

[12]  Ashish Tiwari,et al.  Safety envelope for security , 2014, HiCoNS.

[13]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[14]  Stephen Whitlow,et al.  Assessing V and V Processes for Automation with Respect to Vulnerabilities to Loss of Airplane State Awareness , 2014 .

[15]  Peter J. Bentley,et al.  Autonomous Navigation and Landing of Airliners Using Artificial Neural Networks and Learning by Imitation , 2017 .

[16]  Ananthram Swami,et al.  Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks , 2015, 2016 IEEE Symposium on Security and Privacy (SP).

[17]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[18]  Nikunj C. Oza,et al.  Using ADOPT Algorithm and Operational Data to Discover Precursors to Aviation Adverse Events , 2018 .

[19]  Doina Precup,et al.  Smart exploration in reinforcement learning using absolute temporal difference errors , 2013, AAMAS.

[20]  Chih-Hong Cheng,et al.  Neural networks for safety-critical applications — Challenges, experiments and perspectives , 2017, 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[21]  Myoungho Sunwoo,et al.  Development of Autonomous Car—Part I: Distributed System Architecture and Development Process , 2014, IEEE Transactions on Industrial Electronics.

[22]  John Rushby,et al.  Assurance and Assurance Cases , 2017, Dependable Software Systems Engineering.

[23]  Denis Besnard,et al.  Cognitive mismatches in the cockpit: will they ever be a thing of the past? , 2007, Applied ergonomics.

[24]  Chong Wang,et al.  Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin , 2015, ICML.

[25]  Tegan Maharaj,et al.  Deep Nets Don't Learn via Memorization , 2017, ICLR.

[26]  Eugene Lavretsky,et al.  Adaptive Control and the NASA X-15-3 Flight Revisited , 2010, IEEE Control Systems.

[27]  John C. Knight,et al.  Understanding What It Means for Assurance Cases to "Work" , 2017 .

[28]  Sven Ove Hansson,et al.  Principles of engineering safety: Risk and uncertainty reduction , 2008, Reliab. Eng. Syst. Saf..

[29]  Alwyn E. Goodloe,et al.  Formal analysis of pilot error with agent safety logic , 2018, Innovations in Systems and Software Engineering.

[30]  Anish Arora,et al.  Detectors and correctors: a theory of fault-tolerance components , 1998, Proceedings. 18th International Conference on Distributed Computing Systems (Cat. No.98CB36183).

[31]  Lui Sha,et al.  Using Simplicity to Control Complexity , 2001, IEEE Softw..

[32]  Cliff B. Jones,et al.  Tentative steps toward a development method for interfering programs , 1983, TOPL.

[33]  Niklas Möller.  The Concepts of Risk and Safety , 2012 .

[34]  John M. Rushby.  Verification Diagrams Revisited: Disjunctive Invariants for Easy Verification , 2000, CAV.

[35]  John P. Thomas,et al.  Extending and automating a systems-theoretic hazard analysis for requirements generation and analysis , 2013 .

[36]  Seth Ahrenbach,et al.  FORMAL ANALYSIS OF PILOT ERROR USING AGENT SAFETY LOGIC , 2016 .

[37]  Patrice Micouin.  Model Based Systems Engineering: Fundamentals and Methods , 2014 .

[38]  Sanjit A. Seshia,et al.  Towards Verified Artificial Intelligence , 2016, ArXiv.

[39]  Ronen Marmorstein.  7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER , 2006 .

[40]  C. R. Ramakrishnan,et al.  Compositional Analysis for Verification of Parameterized Systems , 2003, TACAS.

[41]  Ashish Tiwari,et al.  Output Range Analysis for Deep Feedforward Neural Networks , 2018, NFM.

[42]  Seth Ahrenbach.  Reasoning About Safety-Critical Information Flow Between Pilot and Computer , 2017, NFM.

[43]  C. Allen,et al.  Stanford Encyclopedia of Philosophy , 2011 .

[44]  Wladimir Schamai,et al.  Testing of Autonomous Systems – Challenges and Current State‐of‐the‐Art , 2016 .

[45]  Peter J. Bentley,et al.  An Intelligent Autopilot System that learns piloting skills from human pilots by imitation , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).

[46]  C. Michael Holloway.  Making the Implicit Explicit: Towards an Assurance Case for DO-178C , 2013 .

[47]  Radu Calinescu,et al.  Assured Reinforcement Learning for Safety-Critical Applications , 2017 .

[48]  Sagar Behere,et al.  A functional architecture for autonomous driving , 2015, 2015 First International Workshop on Automotive Software Architecture (WASA).

[49]  Patrice Micouin,et al.  Toward a property based requirements theory: System requirements structured as a semilattice , 2008, Syst. Eng..

[50]  Martin S. Feather,et al.  A Hybrid Method of Assurance Cases and Testing for Improved Confidence in Autonomous Space Systems , 2018 .

[51]  Sasikumar Punnekkat,et al.  Improving Dependability of Vision-Based Advanced Driver Assistance Systems Using Navigation Data and Checkpoint Recognition , 2015, SAFECOMP.

[52]  Michael Fisher,et al.  Verifying autonomous systems , 2013, CACM.

[53]  Christian Berger,et al.  Engineering Autonomous Driving Software , 2014, ArXiv.

[54]  Ronen I. Brafman,et al.  R-MAX - A General Polynomial Time Algorithm for Near-Optimal Reinforcement Learning , 2001, J. Mach. Learn. Res..

[55]  Gregory J. Holt.  The Certification Challenge of the High Technology Aircraft of the 1990's , 1987 .

[56]  Alwyn E. Goodloe,et al.  Challenges in the Verification of Reinforcement Learning Algorithms , 2017 .

[57]  Vijay Manikandan Janakiraman.  Explaining Aviation Safety Incidents Using Deep Learned Precursors , 2017, ArXiv.

[58]  Nikunj C. Oza,et al.  Finding Precursors to Anomalous Drop in Airspeed During a Flight's Takeoff , 2017, KDD.

[59]  John M. Rushby,et al.  The Versatile Synchronous Observer , 2012, Specification, Algebra, and Software.

[60]  John J. Grefenstette,et al.  Test and evaluation by genetic algorithms , 1993, IEEE Expert.

[61]  Mykel J. Kochenderfer,et al.  Policy compression for aircraft collision avoidance systems , 2016, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC).

[62]  W. B. Roberts,et al.  Machine Learning: The High Interest Credit Card of Technical Debt , 2014 .

[63]  John Rushby.  PVS Embeddings of Propositional and Quantified Modal Logic , 2018 .

[64]  Michael Fisher,et al.  Formal Methods for the Certification of Autonomous Unmanned Aircraft Systems , 2011, SAFECOMP.

[65]  Daniel Marcu,et al.  Domain Adaptation for Statistical Classifiers , 2006, J. Artif. Intell. Res..

[66]  Kouichi Sakurai,et al.  One Pixel Attack for Fooling Deep Neural Networks , 2017, IEEE Transactions on Evolutionary Computation.

[67]  Lu Feng,et al.  Automated Learning of Probabilistic Assumptions for Compositional Reasoning , 2011, FASE.

[68]  Jaakko Hintikka,et al.  Knowledge and Belief: An Introduction to the Logic of the Two Notions. , 1965 .

[69]  Dov Dori,et al.  Model-Based Systems Engineering with OPM and SysML , 2016, Springer New York.

[70]  Nathan Fulton,et al.  Safe Reinforcement Learning via Formal Methods: Toward Safe Control Through Proof and Learning , 2018, AAAI.

[71]  Gregory M Papadopoulos,et al.  Data flow models for fault-tolerant computation , 1983 .

[72]  Peter J. Bentley,et al.  Autonomous landing and go-around of airliners under severe weather conditions using Artificial Neural Networks , 2017, 2017 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED-UAS).

[73]  Frank Flemisch,et al.  Towards a dynamic balance between humans and automation: authority, ability, responsibility and control in shared and cooperative control situations , 2012, Cognition, Technology & Work.

[74]  Paul Ammann,et al.  Data Diversity: An Approach to Software Fault Tolerance , 1988, IEEE Trans. Computers.

[75]  John Thomas,et al.  STPA-based Method to Identify and Control Feature Interactions in Large Complex Systems☆ , 2015 .

[76]  Andrea Scarinci.  Monitoring safety during airline operations : a systems approach , 2017 .

[77]  Radu Calinescu,et al.  Assured Reinforcement Learning with Formally Verified Abstract Policies , 2017, ICAART.

[78]  J. Blair,et al.  Informal Logic , 2003 .

[79]  Lorenzo Strigini,et al.  Software Fault-Freeness and Reliability Predictions , 2013, SAFECOMP.

[80]  Longbing Cao,et al.  SVDD-based outlier detection on uncertain data , 2012, Knowledge and Information Systems.

[81]  Kush R. Varshney,et al.  On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products , 2016, Big Data.

[82]  Algirdas Avizienis,et al.  The N-Version Approach to Fault-Tolerant Software , 1985, IEEE Transactions on Software Engineering.

[83]  Javier García,et al.  A comprehensive survey on safe reinforcement learning , 2015, J. Mach. Learn. Res..

[84]  Chih-Hong Cheng,et al.  Maximum Resilience of Artificial Neural Networks , 2017, ATVA.

[85]  W. H. F. Barnes.  The Nature of Explanation , 1944, Nature.

[86]  Seiichi Uchida,et al.  A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data , 2016, PloS one.

[87]  Susannah B. F. Paletz,et al.  The Breakdown of Coordinated Decision Making in Distributed Systems , 2010, Hum. Factors.

[88]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[89]  David Wright,et al.  Modeling the probability of failure on demand (pfd) of a 1-out-of-2 system in which one channel is "quasi-perfect" , 2017, Reliab. Eng. Syst. Saf..

[90]  Jacob L. Cybulski,et al.  The Formal and the Informal in Requirements Engineering , 1998 .

[91]  Zhuo Chen,et al.  Improving adherence to heart failure management guidelines via abductive reasoning* , 2017, Theory and Practice of Logic Programming.

[92]  Matthias Scheutz,et al.  A Framework for Developing and Using Shared Mental Models in Human-Agent Teams , 2017 .

[93]  Nancy G. Leveson,et al.  Engineering a Safer World: Systems Thinking Applied to Safety , 2012 .

[94]  Sasikumar Punnekkat,et al.  Improving Intelligent Vehicle Dependability by Means of Infrastructure-Induced Tests , 2015, 2015 IEEE International Conference on Dependable Systems and Networks Workshops.

[95]  Gerald C. Cohen,et al.  Requirements Specification Language (RSL) and supporting tools , 1992 .

[96]  Paulo Tabuada,et al.  Secure State Estimation for Cyber-Physical Systems Under Sensor Attacks: A Satisfiability Modulo Theory Approach , 2014, IEEE Transactions on Automatic Control.

[97]  J. Goldberg,et al.  SIFT: Design and analysis of a fault-tolerant computer for aircraft control , 1978, Proceedings of the IEEE.

[98]  Hoyt Lougee,et al.  SOFTWARE CONSIDERATIONS IN AIRBORNE SYSTEMS AND EQUIPMENT CERTIFICATION , 2001 .

[99]  John Rushby,et al.  On the Interpretation of Assurance Case Arguments , 2015, JSAI-isAI Workshops.

[100]  John Rushby,et al.  Using model checking to help discover mode confusions and other automation surprises , 2002, Reliab. Eng. Syst. Saf..

[101]  Karen M. Feigh,et al.  Example of a Complementary Use of Model Checking and Human Performance Simulation , 2014, IEEE Transactions on Human-Machine Systems.

[102]  Peter J. Bentley,et al.  An Intelligent Autopilot System that learns flight emergency procedures by imitating human pilots , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[103]  Michael Luck,et al.  Evolutionary testing of autonomous software agents , 2009, Autonomous Agents and Multi-Agent Systems.

[104]  Tim Kelly.  A Systematic Approach to Safety Case Management , 2004 .

[105]  George Kesidis,et al.  When Not to Classify: Anomaly Detection of Attacks (ADA) on DNN Classifiers at Test Time , 2017, Neural Computation.

[106]  Corina S. Pasareanu,et al.  Learning Assumptions for Compositional Verification , 2003, TACAS.

[107]  Jonathan M. Ford,et al.  Modular Certification , 2002 .

[108]  Naftali Tishby,et al.  Opening the Black Box of Deep Neural Networks via Information , 2017, ArXiv.

[109]  John M. Rushby,et al.  Runtime Certification , 2008, RV.