2 Short-term Research Priorities 2 . 1 Optimizing AI ’ s Economic Impact

Executive Summary: Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to research how to maximize these benefits while avoiding potential pitfalls. This document gives numerous examples (which should by no means be construed as an exhaustive list) of such worthwhile research aimed at ensuring that AI remains robust and beneficial. Artificial intelligence (AI) research has explored a variety of problems and approaches since its inception, but for the last 20 years or so has been focused on the problems surrounding the construction of intelligent agents – systems that perceive and act in some environment. In this context, the criterion for intelligence is related to statistical and economic notions of rationality – colloquially, the ability to make good decisions, plans, or inferences. The adoption of probabilistic representations and statistical learning methods has led to a large degree of integration and cross-fertilization between AI, machine learning, statistics, control theory, neuroscience, and other fields. The establishment of shared theoretical frameworks, combined with the availability of data and processing power, has yielded remarkable successes in various component tasks such as speech recognition, image classification, autonomous vehicles, machine translation, legged locomotion, and question-answering systems. As capabilities in these areas and others cross the threshold from laboratory research to economically valuable technologies, a virtuous cycle takes hold whereby even small improvements in performance are worth large sums of money, prompting greater investments in research. There is now a broad consensus that AI research is progressing steadily, and that its impact on society is likely to increase. The potential benefits are huge, since everything that civilization has to offer is a product of human intelligence; we cannot predict what we might achieve when this intelligence is magnified by the tools AI may provide, but the eradication of disease and poverty are not unfathomable. Because of the great potential of AI, it is valuable to investigate how to reap its benefits while avoiding potential pitfalls. The progress in AI research makes it timely to focus research not only on making AI more capable, but also on maximizing the societal benefit of AI. Such considerations motivated the AAAI 2008–09 Presidential Panel on Long-Term AI Futures [43] and other projects and community efforts on AI impacts. These constitute a significant expansion of the field of AI itself, which up to now has focused largely on techniques that …

[1]  Laurent Orseau,et al.  Space-Time Embedded Intelligence , 2012, AGI.

[2]  Joseph Y. Halpern,et al.  Game theory with translucent players , 2013, TARK.

[3]  Jonathan M. Smith,et al.  Preliminary design of the SAFE platform , 2011, PLOS '11.

[4]  Michael Norrish,et al.  seL4: formal verification of an OS kernel , 2009, SOSP '09.

[5]  Y. Vanderborght Basic income. An anthology of contemporary research. , 2013 .

[6]  Alan K. Mackworth Agents, Bodies, Constraints, Dynamics, and Evolution , 2009, AI Mag..

[7]  André Platzer,et al.  Logical Analysis of Hybrid Systems - Proving Theorems for Complex Dynamics , 2010 .

[8]  Wei Chu,et al.  Preference learning with Gaussian processes , 2005, ICML.

[9]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[10]  David Andre,et al.  State abstraction for programmable reinforcement learning agents , 2002, AAAI/IAAI.

[11]  Carla E. Brodley,et al.  Machine learning techniques for the computer security domain of anomaly detection , 2000 .

[12]  P. Asaro What Should We Want From a Robot Ethic? , 2020, Machine Ethics and Robot Ethics.

[13]  Jordi Sabater-Mir,et al.  Review on Computational Trust and Reputation Models , 2005, Artificial Intelligence Review.

[14]  Yehuda Lindell,et al.  Privacy Preserving Data Mining , 2002, Journal of Cryptology.

[15]  Rajeev Alur,et al.  Formal verification of hybrid systems , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[16]  Max Tegmark Friendly Artificial Intelligence: The Physics Challenge , 2015, AAAI Workshop: AI and Ethics.

[17]  D. Chalmers The Singularity: a Philosophical Analysis , 2010 .

[18]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[19]  J. Mokyr,et al.  Secular stagnation? Not in your life , 2014 .

[20]  Christopher A. Rouff,et al.  Agent Technology from a Formal Perspective , 2006 .

[21]  Yuriy Brun,et al.  Finding latent code errors via machine learning over program executions , 2004, Proceedings. 26th International Conference on Software Engineering.

[22]  Carsten Willems,et al.  Automatic analysis of malware behavior using machine learning , 2011, J. Comput. Secur..

[23]  Benja Fallenstein,et al.  Questions of Reasoning Under Logical Uncertainty , 2015 .

[24]  Stephen Omohundro,et al.  The Nature of Self-Improving Artificial Intelligence , 2008 .

[25]  Matthew C. Waxman,et al.  Adapting the Law of Armed Conflict to Autonomous Weapon Systems , 2014 .

[26]  N. Weaver Paradoxes of rational agency and formal systems that verify their own soundness , 2013, 1312.3626.

[27]  Bill Hibbard,et al.  Ethical Artificial Intelligence , 2014, Mar/Apr 2020.

[28]  Sneha Kumar Kasera,et al.  Statistical trust establishment in wireless sensor networks , 2007, 2007 International Conference on Parallel and Distributed Systems.

[29]  Nate Soares,et al.  Formalizing Two Problems of Realistic World-Models , 2015 .

[30]  Bonnie Docherty,et al.  Losing Humanity : The Case Against Killer Robots , 2012 .

[31]  Oren Etzioni,et al.  The First Law of Robotics (A Call to Arms) , 1994, AAAI.

[32]  Yan Liu,et al.  Applications of Neural Networks in High Assurance Systems , 2010, Applications of Neural Networks in High Assurance Systems.

[33]  The Scientists’ Call to Ban Autonomous Lethal Robots , 2019, Militarization.

[34]  Bill Hibbard,et al.  Avoiding Unintended AI Behaviors , 2012, AGI.

[35]  Christian Blum,et al.  Towards an Ethical Robot: Internal Models, Consequences and Ethical Action Selection , 2014, TAROS.

[36]  Joseph Y. Halpern,et al.  Decision Theory with Resource-Bounded Agents , 2014, Top. Cogn. Sci..

[37]  R. Churchill,et al.  Autonomous Institutional Arrangements in Multilateral Environmental Agreements: A Little-Noticed Phenomenon in International Law , 2000, American Journal of International Law.

[38]  Bruce M. McLaren,et al.  Computational Models of Ethical Reasoning: Challenges, Initial Steps, and Future Directions , 2006, IEEE Intelligent Systems.

[39]  Michael Fisher,et al.  Practical verification of decision-making in agent-based autonomous systems , 2013, Automated Software Engineering.

[40]  Nick Bostrom,et al.  The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents , 2012, Minds and Machines.

[41]  Luca Pulina,et al.  An Abstraction-Refinement Approach to Verification of Artificial Neural Networks , 2010, CAV.

[42]  Nils J. Nilsson,et al.  Artificial Intelligence, Employment, and Income , 1984, AI Mag..

[43]  Kristian J. Hammond,et al.  The Stabilization of Environments , 1995, Artif. Intell..

[44]  Henry Hexmoor,et al.  Natural human role in supervising complex control systems , 2009, J. Exp. Theor. Artif. Intell..

[45]  Heather Roff,et al.  The Strategic Robot Problem: Lethal Autonomous Weapons in War , 2014 .

[46]  Brian J. Taylor,et al.  Methods and Procedures for the Verification and Validation of Artificial Neural Networks , 2005 .

[47]  J. Manyika,et al.  Disruptive technologies: Advances that will transform life, business, and the global economy , 2013 .

[48]  B. Selman,et al.  Interim Report from the Panel Chairs: AAAI Presidential Panel on Long-Term AI Futures , 2012 .

[49]  J. Manyika Big data: The next frontier for innovation, competition, and productivity , 2011 .

[50]  Clemens Hetschko,et al.  Changing Identity: Retiring from Unemployment , 2011, SSRN Electronic Journal.

[51]  C. Allen,et al.  Moral Machines: Teaching Robots Right from Wrong , 2008 .

[52]  D. Vladeck Machines without Principals: Liability Rules and Artificial Intelligence , 2014 .

[53]  Roman V. Yampolskiy,et al.  Leakproofing the Singularity Artificial Intelligence Confinement Problem , 2012 .

[54]  Diana F. Spears,et al.  Assuring the Behavior of Adaptive Agents , 2006 .

[55]  Devika Subramanian,et al.  Provably Bounded Optimal Agents , 1993, IJCAI.

[56]  Joseph Y. Halpern I Don't Want to Think About it Now: Decision Theory with Costly Computation , 2010, KR.

[57]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[58]  Martin Kiefel,et al.  Quasi-Newton Methods: A New Direction , 2012, ICML.

[59]  Luís Moniz Pereira,et al.  Modelling morality with prospective logic , 2007, Int. J. Reason. based Intell. Syst..

[60]  Peter Asaro,et al.  How Just Could a Robot War Be , 2008 .

[61]  Ryan Calo The Case for a Federal Robotics Commission , 2014 .

[62]  A. Oswald,et al.  Unhappiness and Unemployment , 1994 .

[63]  B. Pasik-Duncan,et al.  Adaptive Control , 1996, IEEE Control Systems.

[64]  Benja Fallenstein,et al.  Program Equilibrium in the Prisoner ’ s Dilemma via Löb ’ s Theorem , 2014 .

[65]  Michael A. Osborne,et al.  The future of employment: How susceptible are jobs to computerisation? , 2017 .

[66]  Roman Garnett,et al.  Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature , 2014, NIPS.

[67]  I. J. Good,et al.  Speculations Concerning the First Ultraintelligent Machine , 1965, Adv. Comput..

[68]  Benja Fallenstein,et al.  Aligning Superintelligence with Human Interests: A Technical Research Agenda , 2015 .

[69]  Eric Horvitz,et al.  Reasoning about beliefs and actions under computational resource constraints , 1987, Int. J. Approx. Reason..

[70]  C. Robert Superintelligence: Paths, Dangers, Strategies , 2017 .

[71]  Moshe Tennenholtz,et al.  Program equilibrium , 2004, Games Econ. Behav..

[72]  A D Wissner-Gross,et al.  Causal entropic forces. , 2013, Physical review letters.

[73]  Bill Hibbard Self-Modeling Agents and Reward Generator Corruption , 2015, AAAI Workshop: AI and Ethics.

[74]  V. Vinge The coming technological singularity , 1993 .

[75]  Michael Marien,et al.  Book Review: The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies , 2014 .

[76]  P. Parijs Arguing for Basic Income: Ethical Foundations for a Radical Reform , 1992 .

[77]  Daniel Hintze Problem Class Dominance in Predictive Dilemmas , 2014 .

[78]  John P. Sullins Introduction: Open Questions in Roboethics , 2011 .

[79]  Kathleen Fisher HACMS: high assurance cyber military systems , 2012, HILT.