The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

The following organisations are named on the report: Future of Humanity Institute, University of Oxford, Centre for the Study of Existential Risk, University of Cambridge, Center for a New American Security, Electronic Frontier Foundation, OpenAI. The Future of Life Institute is acknowledged as a funder.

[1]  Erik Brynjolfsson,et al.  The second machine age: work, progress, and prosperity in a time of brilliant technologies, 1st Edition , 2014 .

[2]  Eric Horvitz,et al.  Social media as a measurement tool of depression in populations , 2013, WebSci.

[3]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[4]  David L. Dill,et al.  Developing Bug-Free Machine Learning Systems With Formal Mathematics , 2017, ICML.

[5]  Dennis M. Murphy.  The Net Delusion: The Dark Side of Internet Freedom , 2012 .

[6]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[7]  Bill Fitzgerald.  Facebook Tinkers With Users’ Emotions in News Feed Experiment, Stirring Outcry , 2015 .

[8]  Meaningful Human Control , Artificial Intelligence and Autonomous Weapons Briefing paper for delegates at the Convention on Certain Conventional Weapons , 2022 .

[9]  Joon Son Chung,et al.  You said that? , 2017, BMVC.

[10]  Davide Cozzolino,et al.  Autoencoder with recurrent neural networks for video forgery detection , 2017, Media Watermarking, Security, and Forensics.

[11]  Alvin Zhou.  #Republic: Divided Democracy in the Age of Social Media , 2017 .

[12]  Ming-Yu Liu,et al.  Coupled Generative Adversarial Networks , 2016, NIPS.

[13]  Nils B. Weidmann,et al.  Empowering activists or autocrats? The Internet in authoritarian regimes , 2015 .

[14]  B. Turner.  Center for a New American Security , 2014 .

[15]  Tom Schaul,et al.  FeUdal Networks for Hierarchical Reinforcement Learning , 2017, ICML.

[16]  Andrew Chi-Chih Yao,et al.  Protocols for secure computations , 1982, FOCS 1982.

[17]  Karim K. Shehadeh.  The Wassenaar Arrangement and Encryption Exports: An Ineffective Export Control Regime that Compromises United States' Economic Interests , 1999 .

[18]  Geoffrey R. Stone,et al.  Liberty and Security in a Changing World: Report and Recommendations of The President's Review Group on Intelligence and Communications Technologies , 2013 .

[19]  Ryan Calo.  Robotics and the Lessons of Cyberlaw , 2014 .

[20]  Jeffrey T. Hancock,et al.  Editorial Expression of Concern and Correction , 2014 .

[21]  Filippo Menczer,et al.  The spread of fake news by social bots , 2017, ArXiv.

[22]  Ananthram Swami,et al.  Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples , 2016, ArXiv.

[23]  Eran Tromer,et al.  PhotoProof: Cryptographic Image Authentication for Any Set of Permissible Transformations , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[24]  Radha Poovendran,et al.  Google's Cloud Vision API is Not Robust to Noise , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[25]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[26]  Michael P. Wellman,et al.  Towards the Science of Security and Privacy in Machine Learning , 2016, ArXiv.

[27]  John Harrison.  Formal Methods at Intel - An Overview , 2010 .

[28]  Mahmudur Rahman,et al.  Video Liveness for Citizen Journalism: Attacks and Defenses , 2017, IEEE Transactions on Mobile Computing.

[29]  Mary L. Cummings,et al.  Creating moral buffers in weapon control interface design , 2004, IEEE Technology and Society Magazine.

[30]  Jeffrey T. Hancock,et al.  Experimental evidence of massive-scale emotional contagion through social networks , 2014, Proceedings of the National Academy of Sciences.

[31]  Ross Anderson,et al.  Internet Security , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[32]  Jason R. C. Nurse,et al.  The anatomy of online deception: what makes automated text convincing? , 2016, SAC.

[33]  Patrick D. McDaniel,et al.  Cleverhans V0.1: an Adversarial Machine Learning Library , 2016, ArXiv.

[34]  Abhishek Kashyap,et al.  An Evaluation of Digital Image Forgery Detection Approaches , 2017, ArXiv.

[35]  Yoshua Bengio,et al.  NICE: Non-linear Independent Components Estimation , 2014, ICLR.

[36]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[37]  Mark S. Miller,et al.  Robust composition: towards a unified approach to access control and concurrency control , 2006 .

[38]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[39]  Mike Preuss,et al.  Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Network Policies , 2017, ICLR.

[40]  Moni Naor,et al.  Pricing via Processing or Combatting Junk Mail , 1992, CRYPTO.

[41]  Joanna Bryson,et al.  Of, for, and by the people: the legal lacuna of synthetic persons , 2017, Artificial Intelligence and Law.

[42]  José Hernández-Orallo,et al.  The Measure of All Minds: Evaluating Natural and Artificial Intelligence , 2017 .

[43]  Daniele Quercia,et al.  Our Twitter Profiles, Our Selves: Predicting Personality with Twitter , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[44]  Suhang Wang,et al.  Fake News Detection on Social Media: A Data Mining Perspective , 2017, SKDD.

[45]  Kathleen Fisher.  Using formal methods to enable more secure vehicles: DARPA's HACMS program , 2014, ICFP.

[46]  Thomas Zeitzoff,et al.  How Social Media Is Changing Conflict , 2017 .

[47]  Brendan Dolan-Gavitt,et al.  BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain , 2017, ArXiv.

[48]  Juliane Hahn,et al.  Security And Game Theory Algorithms Deployed Systems Lessons Learned , 2016 .

[49]  Charles Elkan,et al.  Differential Privacy and Machine Learning: a Survey and Review , 2014, ArXiv.

[50]  Verónica Pérez-Rosas,et al.  Automatic Detection of Fake News , 2017, COLING.

[51]  Cormac Herley,et al.  The Plight of the Targeted Attacker in a World of Scale , 2010, WEIS.

[52]  Margaret E. Roberts,et al.  How the Chinese Government Fabricates Social Media Posts for Strategic Distraction, Not Engaged Argument , 2017, American Political Science Review.

[53]  Farinaz Koushanfar,et al.  DeepSecure: Scalable Provably-Secure Deep Learning , 2017, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[54]  Ryan Calo,et al.  There is a blind spot in AI research , 2016, Nature.

[55]  Percy Liang,et al.  Understanding Black-box Predictions via Influence Functions , 2017, ICML.

[56]  Ben Laurie,et al.  “ Proof-of-Work ” Proves Not to Work version 0 . 2 , 2004 .

[57]  Catherine Jefferson.  Innovation, Dual Use, and Security: Managing the Risks of Emerging Biological and Chemical Technologies , 2012 .

[58]  Justus Thies,et al.  Face2Face: real-time face capture and reenactment of RGB videos , 2019, Commun. ACM.

[59]  Micah Sherr,et al.  Hidden Voice Commands , 2016, USENIX Security Symposium.

[60]  Samuel C. Woolley,et al.  Computational propaganda worldwide: Executive summary , 2017 .

[61]  Edsger W. Dijkstra,et al.  Letters to the editor: go to statement considered harmful , 1968, CACM.

[62]  Sebastian Farquhar,et al.  Pricing Externalities to Balance Public Risks and Benefits of Research , 2017, Health security.

[63]  Shashank V. Joshi,et al.  Army of none: autonomous weapons and the future of war , 2018, International Affairs.

[64]  L. Jean Camp,et al.  Proof of Work can Work , 2006, WEIS.

[65]  T. Graepel,et al.  Private traits and attributes are predictable from digital records of human behavior , 2013, Proceedings of the National Academy of Sciences.

[66]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[67]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[68]  Seth Flaxman,et al.  European Union Regulations on Algorithmic Decision-Making and a "Right to Explanation" , 2016, AI Mag..

[69]  Ling Huang,et al.  ANTIDOTE: understanding and defending against poisoning of anomaly detectors , 2009, IMC '09.

[70]  Arkaitz Zubiaga,et al.  Detection and Resolution of Rumours in Social Media , 2017, ACM Comput. Surv..

[71]  Pavel Laskov,et al.  Practical Evasion of a Learning-Based Classifier: A Case Study , 2014, 2014 IEEE Symposium on Security and Privacy.

[72]  Tom Schaul,et al.  Reinforcement Learning with Unsupervised Auxiliary Tasks , 2016, ICLR.

[73]  Somesh Jha,et al.  Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures , 2015, CCS.

[74]  John Salvatier,et al.  When Will AI Exceed Human Performance? Evidence from AI Experts , 2017, ArXiv.

[75]  Randy H. Katz,et al.  A Berkeley View of Systems Challenges for AI , 2017, ArXiv.

[76]  Rand Waltzman,et al.  The Weaponization of Information: The Need for Cognitive Security , 2017 .

[77]  Tom Schaul,et al.  Deep Q-learning From Demonstrations , 2017, AAAI.

[78]  Jon Turney,et al.  Molecular politics: developing American and British regulatory policy for genetic engineering, 1972–1982 , 1996, Medical History.

[79]  Patrick D. McDaniel,et al.  Adversarial Perturbations Against Deep Neural Networks for Malware Classification , 2016, ArXiv.

[80]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[81]  William E. Kastenberg,et al.  Making ethics explicit: Relocating ethics to the core of engineering education , 2013 .

[82]  Bobby Filar,et al.  Ask Me Anything: A Conversational Interface to Augment Information Security Workers , 2017, SOUPS.

[83]  Joelle Pineau,et al.  A Deep Reinforcement Learning Chatbot , 2017, ArXiv.

[84]  Jaakko Lehtinen,et al.  Progressive Growing of GANs for Improved Quality, Stability, and Variation , 2017, ICLR.

[85]  Tung Yin.  Game of Drones: Defending Against Drone Terrorism , 2015 .

[86]  Rebecca Crootof,et al.  The Killer Robots Are Here: Legal and Policy Implications , 2014 .

[87]  Thomas Wollinger,et al.  IT Security and the Internet of Things , 2014 .

[88]  Vijay Arya,et al.  Model Extraction Warning in MLaaS Paradigm , 2017, ACSAC.

[89]  W. Nuland,et al.  Information operations and Facebook , 2017 .

[90]  Vitaly Shmatikov,et al.  Membership Inference Attacks Against Machine Learning Models , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[91]  Laurent Orseau,et al.  Reinforcement Learning with a Corrupted Reward Channel , 2017, IJCAI.

[92]  Martin Libicki.  Cyberspace in Peace and War , 2016 .

[93]  Eleonora Mattiacci,et al.  A Manifesto, in 140 Characters or Fewer: Social Media as a Tool of Rebel Diplomacy , 2017, British Journal of Political Science.

[94]  Sheldon Krimsky,et al.  Genetic Alchemy: The Social History of the Recombinant DNA Controversy , 1984 .

[95]  Yehuda Lindell,et al.  Secure Multiparty Computation for Privacy-Preserving Data Mining , 2009, IACR Cryptol. ePrint Arch..

[96]  P. W. Singer,et al.  Wired for War: The Robotics Revolution and Conflict in the 21st Century , 2009 .

[97]  Tudor Dumitras,et al.  Summoning Demons: The Pursuit of Exploitable Bugs in Machine Learning , 2017, ArXiv.

[98]  B. B. D. Mesquita,et al.  The Dictator's Handbook: Why Bad Behavior is Almost Always Good Politics , 2011 .

[99]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.