Predicting future AI failures from historic examples

Purpose The purpose of this paper is to explain to readers how intelligent systems can fail and how artificial intelligence (AI) safety is different from cybersecurity. The goal of cybersecurity is to reduce the number of successful attacks on the system; the goal of AI Safety is to make sure zero attacks succeed in bypassing the safety mechanisms. Unfortunately, such a level of performance is unachievable. Every security system will eventually fail; there is no such thing as a 100 per cent secure system. Design/methodology/approach AI Safety can be improved based on ideas developed by cybersecurity experts. For narrow AI Safety, failures are at the same, moderate level of criticality as in cybersecurity; however, for general AI, failures have a fundamentally different impact. A single failure of a superintelligent system may cause a catastrophic event without a chance for recovery. Findings In this paper, the authors present and analyze reported failures of artificially intelligent systems and extrapolate our analysis to future AIs. The authors suggest that both the frequency and the seriousness of future AI failures will steadily increase. Originality/value This is a first attempt to assemble a public data set of AI failures and is extremely valuable to AI Safety researchers.

[1]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[2]  James R. Meehan,et al.  TALE-SPIN, An Interactive Program that Writes Stories , 1977, IJCAI.

[3]  W R Hartston.  Artificial stupidity , 1986 .

[4]  G Macpherson,et al.  A blot on the profession , 1988, British medical journal.

[5]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[6]  M. G. Rodd.  Safe AI—is this possible?☆ , 1995 .

[7]  Helen Nissenbaum,et al.  Bias in computer systems , 1996, TOIS.

[8]  Andrew Y. Ng,et al.  Policy Invariance Under Reward Transformations: Theory and Application to Reward Shaping , 1999, ICML.

[9]  Milind Tambe,et al.  Electric Elves: What Went Wrong and Why , 2006, AAAI Spring Symposium: What Went Wrong and Why: Lessons from AI Research and Applications.

[10]  Andreas Abecker,et al.  AAAI 2006 Spring Symposium Reports , 2006, AI Mag..

[11]  Mehmet H. Göker,et al.  What went wrong and why : lessons from AI research and applications : papers from the AAAI Spring Symposium , 2006 .

[12]  Eliezer Yudkowsky.  Artificial Intelligence as a Positive and Negative Factor in Global Risk , 2006 .

[13]  James H. Moor,et al.  The Nature, Importance, and Difficulty of Machine Ethics , 2006, IEEE Intelligent Systems.

[14]  Jonathan Lyons.  Artificial stupidity , 2007, SIGGRAPH '07.

[15]  Raffaello D'Andrea,et al.  Coordinating Hundreds of Cooperative, Autonomous Vehicles in Warehouses , 2007, AI Mag..

[16]  Mehmet H. Göker,et al.  Advancing AI Research and Applications by Learning from What Went Wrong and Why , 2008, AI Mag..

[17]  D. Chelberg,et al.  RoboCup for the Mechanically , Athletically and Culturally Challenged , 2008 .

[18]  J. A. Robinson.  Development of Logic Programming : What went wrong , 2008 .

[19]  Armin Krishnan,et al.  Killer Robots: Legality and Ethicality of Autonomous Weapons , 2009 .

[20]  Roman V. Yampolskiy,et al.  Artificial Intelligence Safety Engineering: Why Machine Ethics Is a Wrong Approach , 2011, PT-AI.

[21]  Eliezer Yudkowsky,et al.  Complex Value Systems in Friendly AI , 2011, AGI.

[22]  Roman V. Yampolskiy,et al.  Efficiency Theory : a Unifying Theory for Information, Computation and Intelligence , 2011, Journal of Discrete Mathematical Sciences and Cryptography.

[23]  Daniel Dewey,et al.  Learning What to Value , 2011, AGI.

[24]  Roman V Yampolskiy,et al.  Safety Engineering for Artificial General Intelligence , 2012, Topoi.

[25]  Roman V. Yampolskiy,et al.  AI-Complete CAPTCHAs as Zero Knowledge Proofs of Access to an Artificially Intelligent System , 2012 .

[26]  Nick Bostrom,et al.  Thinking Inside the Box: Controlling and Using an Oracle AI , 2012, Minds and Machines.

[27]  Roman V. Yampolskiy,et al.  Leakproofing the Singularity Artificial Intelligence Confinement Problem , 2012 .

[28]  Roman V. Yampolskiy,et al.  AI-Complete, AI-Hard, or AI-Easy - Classification of Problems in AI , 2012, MAICS.

[29]  Poul-Henning Kamp.  More Encryption Is Not the Solution , 2013, ACM Queue.

[30]  Dr. Tom Murphy.  The First Level of Super Mario Bros . is Easy with Lexicographic Orderings and Time Travel , 2013 .

[31]  Latanya Sweeney,et al.  Discrimination in online ad delivery , 2013, CACM.

[32]  Roman V. Yampolskiy,et al.  Turing Test as a Defining Feature of AI-Completeness , 2013, Artificial Intelligence, Evolutionary Computing and Metaheuristics.

[33]  Latanya Sweeney,et al.  Discrimination in Online Ad Delivery , 2013, ACM Queue.

[34]  Roman V Yampolskiy,et al.  AI safety engineering through introduction of self-reference into felicific calculus via artificial pain and pleasure , 2014, 2014 IEEE International Symposium on Ethics in Science, Technology and Engineering.

[35]  Nick Bostrom,et al.  Superintelligence: Paths, Dangers, Strategies , 2014 .

[36]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[37]  Johannes Gehrke,et al.  Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission , 2015, KDD.

[38]  Roman V Yampolskiy,et al.  Responses to catastrophic AGI risk: a survey , 2014 .

[39]  Roman V. Yampolskiy,et al.  Unethical Research: How to Create a Malevolent Artificial Intelligence , 2016, ArXiv.

[40]  James Babcock,et al.  The AGI Containment Problem , 2016, AGI.

[41]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[42]  Roman V. Yampolskiy,et al.  On the origin of synthetic life: attribution of output to a particular algorithm , 2016, ArXiv.

[43]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[44]  Stuart Armstrong,et al.  Security Solutions for Intelligent and Complex Systems , 2020, Natural Language Processing.

[45]  Roman V. Yampolskiy,et al.  What are the ultimate limits to computational techniques: verifier theory and unverifiability , 2017 .

[46]  S. Baum.  A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy , 2017 .

[47]  Ohad Shamir,et al.  Failures of Gradient-Based Deep Learning , 2017, ICML.

[48]  Arvind Narayanan,et al.  Semantics derived automatically from language corpora contain human-like biases , 2016, Science.

[49]  Roman V Yampolskiy,et al.  BEYOND MAD ? : THE RACE FOR ARTIFICIAL GENERAL INTELLIGENCE , 2018 .

[50]  Hyrum S. Anderson,et al.  The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation , 2018, ArXiv.

[51]  Nick Bostrom,et al.  Ethical Issues in Advanced Artificial Intelligence , 2020 .