Autonomous Systems: Issues for Defence Policymakers

Autonomous systems pose both an opportunity and a challenge for warfighters. Militaries who harness their advantages will be able to field forces with greater range, persistence, mass, coordination, and speed on the battlefield. At the same time, autonomous systems, in their current state, lack the robustness and flexibility of human intelligence and are not appropriate for many tasks. Human-machine teaming to combine the best of each is likely to result in the most capable future force. But determining the appropriate balance between human and machine cognition will not always be easy. Autonomous systems are already capable enough to eliminate many tasks currently performed by warfighters, and are likely to continue improving over time. At the same time, war is and will remain a human endeavour. Robots are not combatants, but are merely tools in the hands of warfighters to better perform their job: to fight and win. Introduction Whether in cars, airplanes, stock-trading algorithms, or household appliances, machines across a range of industries are incorporating increasing intelligence and autonomy. Automated algorithms set book prices online.(1) Subway repairs are scheduled by tailored artificial intelligences.(2) ‘Learning’ thermostats monitor your behaviour and adjust household temperature accordingly.(3) And an estimated 61% of internet traffic is generated by automated and intelligent software called ‘bots’, from so-called Twitter bots that impersonate

[1]  Michael Wooldridge,et al.  Game Theory and Decision Theory in Multi-Agent Systems , 2002, Autonomous Agents and Multi-Agent Systems.

[2]  Frank Douma,et al.  Self-Driving Vehicles: Current Status of Autonomous Vehicle Development and Minnesota Policy Implications , 2014 .

[3]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[4]  Heather Roff,et al.  The Strategic Robot Problem: Lethal Autonomous Weapons in War , 2014 .

[5]  Timothy,et al.  International Soft Law , 2010 .

[6]  Ronald C. Arkin,et al.  International Governance of Autonomous Military Robots , 2010 .

[7]  Bernd Carsten Stahl,et al.  Emancipation in cross-cultural IS research: The fine line between relativism and dictatorship of the intellectual , 2006, Ethics and Information Technology.

[8]  Geert-Jan Alexander Knoops Legal, Political and Ethical Dimensions of Drone Warfare under International Law: A Preliminary Survey , 2012 .

[9]  I. Lakatos II-CRITICISM AND THE METHODOLOGY OF SCIENTIFIC RESEARCH PROGRAMMES , 1969 .

[10]  W. A. Olson,et al.  Autonomy based human-vehicle interface standards for remotely operated aircraft , 2001, 20th DASC. 20th Digital Avionics Systems Conference (Cat. No.01CH37219).

[11]  Sandor M. Veres,et al.  Documents for Intelligent Agents in English , 2010 .

[12]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[13]  John D Lee,et al.  Dynamics of Driver Distraction: The process of engaging and disengaging. , 2014, Annals of advances in automotive medicine. Association for the Advancement of Automotive Medicine. Annual Scientific Conference.

[14]  Viswanath Venkatesh,et al.  Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model , 2000, Inf. Syst. Res..

[15]  Julia Spiegel,et al.  Which Law Governs During Armed Conflict? The Relationship Between International Humanitarian Law and Human Rights Law , 2012 .

[16]  Uri Wilensky,et al.  NetLogo: A simple environment for modeling complexity , 2014 .

[17]  Oona A. Hathaway Do Human Rights Treaties Make a Difference , 2002 .

[18]  Patrick T. Hester,et al.  Applying methods of the M&S spectrum for complex systems engineering , 2010, SpringSim.

[19]  Matthew W. M. Edwards,et al.  Establishing a Risk-Based Separation Standard for Unmanned Aircraft Self Separation , 2011 .

[20]  D. Shoup The High Cost of Free Parking , 1997 .

[21]  Jaana Leikas,et al.  Life-Based Design: A holistic approach to designing human-technology interaction , 2009 .

[22]  Nidhi Kalra,et al.  Autonomous Vehicle Technology: A Guide for Policymakers , 2014 .

[23]  Eric Bonabeau,et al.  Agent-based modeling: Methods and techniques for simulating human systems , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Jaana Leikas,et al.  Managing Life-Based Design as Micro-Innovation Process , 2012 .

[25]  Oona A. Hathaway Path Dependence in the Law: The Course and Pattern of Legal Change in a Common Law System , 2003 .

[26]  Neville Moray,et al.  The Good, the Bad, and the Future: On the Archaeology of Ergonomics , 2008, Hum. Factors.

[27]  Roland Siegwart,et al.  Introduction to Autonomous Mobile Robots , 2004 .

[28]  Farid Kendoul,et al.  Towards a Unified Framework for UAS Autonomy and Technology Readiness Assessment (ATRA) , 2013 .

[29]  Larry McKee Latency and Its Effects on the Fidelity of Air-to-Air Missile T&E Using Advanced Distributed Simulations , 1998 .

[30]  D. Kahneman Thinking, Fast and Slow , 2011 .

[31]  Sandra L. Berning,et al.  Adaptive Tactical Navigation Program , 1991 .

[32]  Thomas R. Gruber,et al.  Toward principles for the design of ontologies used for knowledge sharing? , 1995, Int. J. Hum. Comput. Stud..

[33]  H. Freud Emotional Design Why We Love Or Hate Everyday Things , 2016 .

[34]  Todd Litman,et al.  Autonomous Vehicle Implementation Predictions: Implications for Transport Planning , 2015 .

[35]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[36]  Bin Cheng,et al.  Studies In International Space Law , 1997 .

[37]  Rebecca Crootof,et al.  The Killer Robots Are Here: Legal and Policy Implications , 2014 .

[38]  David C. Slaughter,et al.  Autonomous robotic weed control systems: A review , 2008 .

[39]  Andreas Tolk,et al.  Agents: Agenthood, Agent Architectures, and Agent Taxonomies , 2010 .

[40]  Michael Siegrist,et al.  How a Nuclear Power Plant Accident Influences Acceptance of Nuclear Power: Results of a Longitudinal Study Before and After the Fukushima Disaster , 2013, Risk analysis : an official publication of the Society for Risk Analysis.

[41]  Maja J. Mataric,et al.  The Robotics Primer , 2007 .

[42]  John C. McCarthy,et al.  Technology as experience , 2004, INTR.

[43]  Markus Wagner,et al.  Autonomy in the Battlespace: Independently Operating Weapon Systems and the Law of Armed Conflict , 2012 .

[44]  Allyn L Taylor,et al.  Twenty-first century international lawmaking for alcohol control. , 2013, Addiction.

[45]  Charles E. Billings,et al.  Toward a Human-Centered Aircraft Automation Philosophy , 1991 .

[46]  Jaana Leikas,et al.  Life-Based Design - An Approach to Design for Life , 2010 .

[47]  Heather Bradshaw-Martin,et al.  Autonomous or ‘driverless’ cars and disability: a legal and ethical analysis. , 2014 .

[48]  T Palmer Michael,et al.  A Crew-Centered Flight Deck Design Philosophy for High-Speed Civil Transport (HSCT) Aircraft , 1995 .

[49]  P. Saariluoma Foundational Analysis: Presuppositions in Experimental Psychology , 1997 .

[50]  R. J. Kelly,et al.  REQUIRED NAVIGATION PERFORMANCE (RNP) FOR PRECISION APPROACH AND LANDING WITH GNSS APPLICATION , 1994 .

[51]  Renwick E. Curry,et al.  Flight-deck automation: promises and problems , 1980 .

[52]  Ernest H. Page,et al.  Architectural principles for the U.S. Army's simulation and modeling for acquisition, requirements and training (SMART) initiative , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[53]  Noel E. Sharkey Automating warfare: Lessons learned from the drones , 2011 .

[54]  Halimahtun M. Khalid,et al.  Customer Emotional Needs in Product Design , 2006, Concurr. Eng. Res. Appl..

[55]  E. Wilmshurst,et al.  State Immunity: The United Nations Convention and its effect , 2005 .

[56]  Andreas Tolk,et al.  Toward a Research Agenda for M&S Support of System of Systems Engineering , 2014 .

[57]  Ronald C. Arkin,et al.  Governing lethal behavior: Embedding ethics in a hybrid deliberative/reactive robot architecture part I: Motivation and philosophy , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[58]  William H. Boothby,et al.  The Law of Targeting , 2012, The Military Law and the Law of War Review.

[59]  K. Keller,et al.  Pilot's associate - An inflight mission planning application , 1989 .

[60]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[61]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[62]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[63]  William B. Rouse,et al.  Specification of adaptive aiding systems , 1990, 1990 IEEE International Conference on Systems, Man, and Cybernetics Conference Proceedings.

[64]  Scott J. Shackelford From Nuclear War to Net War: Analogizing Cyber Attacks in International Law , 2009 .

[65]  Michael N. Schmitt,et al.  Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics , 2012 .

[66]  Philip S. BARRY,et al.  Agent-Directed Simulation for Systems Engineering : Applications to the Design of Venue Defense and the Oversight of Financial Markets , 2009 .

[67]  Thomas P. Moran,et al.  Guest Editor's Introduction: An Applied Psychology of the User , 1981, CSUR.

[68]  T. Schelling Models of Segregation , 1969 .

[69]  Christopher D. Wickens,et al.  A model for types and levels of human interaction with automation , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[70]  Andreas Tolk,et al.  Towards a taxonomy of agents and multi-agent systems , 2007, SpringSim '07.

[71]  Mike Kuniavsky,et al.  Observing the User Experience: A Practitioner's Guide to User Research (Second Edition) , 2013, IEEE Transactions on Professional Communication.

[72]  Angelika Foerster,et al.  Designing Pleasurable Products An Introduction To The New Human Factors , 2016 .

[73]  Andreas Tolk,et al.  Executable architectures in executable context enabling fit-for-purpose and portfolio assessment , 2015 .

[74]  Elena R. Messina,et al.  Autonomy Level Specification for Intelligent Autonomous Vehicles , 2003 .

[75]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[76]  Bruce Clough,et al.  Metrics, Schmetrics! How Do You Track a UAV's Autonomy? , 2002 .

[77]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[78]  Michael Wooldridge,et al.  Reasoning about rational agents , 2000, Intelligent robots and autonomous agents.

[79]  Brandon Suarez,et al.  From spatial conflict probes to spatial/temporal conflict probes: Why and how , 2013, 2013 IEEE/AIAA 32nd Digital Avionics Systems Conference (DASC).

[80]  Mark W. Maier,et al.  The Role of Modeling and Simulation in System of Systems Development , 2014 .

[81]  Alexander M. Millkey The Black Swan: The Impact of the Highly Improbable , 2009 .

[82]  G.W.M. Rauterberg,et al.  Ambient Culture: A Possible Future for Entertainment Computing , 2007 .

[83]  Nidhi Kalra,et al.  Liability and Regulation of Autonomous Vehicle Technologies , 2009 .

[84]  Hoyt Lougee,et al.  SOFTWARE CONSIDERATIONS IN AIRBORNE SYSTEMS AND EQUIPMENT CERTIFICATION , 2001 .

[85]  B. Turner Center for a New American Security , 2014 .

[86]  Andreas Tolk,et al.  A method for introducing artificial perception (ap) to improve human behavior representation (hbr) using agents in synthetic environments , 2009 .

[87]  Paul B. Larsen,et al.  Space Law: A Treatise , 2007 .

[88]  Clarisse Sieckenius de Souza,et al.  The Semiotic Engineering of Human-Computer Interaction , 2005 .

[89]  Waldemar Karwowski,et al.  The Discipline of Ergonomics and Human Factors , 2006 .

[90]  Jeffrey S. Thurnher The Law that Applies to Autonomous Weapon Systems , 2013 .

[91]  Thomas B. Sheridan,et al.  Human and Computer Control of Undersea Teleoperators , 1978 .

[92]  Antti Oulasvirta,et al.  User Psychology: Re-assessing the Boundaries of a Discipline , 2010 .

[93]  John Charnley Presidential Address: Navigation Aids to Aircraft All-weather Landing , 1989 .

[94]  H. Hachiya Engineering ethics , 2006, Journal of Medical Ultrasonics.

[95]  P. Asaro On banning autonomous weapon systems: human rights, automation, and the dehumanization of lethal decision-making , 2012, International Review of the Red Cross.

[96]  J. Witt,et al.  Two Conceptions of Suffering in War , 2012 .

[97]  R. Shiffrin,et al.  Automatic and controlled processing revisited. , 1984, Psychological review.

[98]  A. Roadmapof A Roadmap of Agent Research and Development , 1995 .

[99]  M. Akehurst,et al.  Custom as a Source of International Law , 1976 .

[100]  H. Champion,et al.  Technical Evaluation Report (HFM Panel Symposium) , 2004 .

[101]  Pertti Saariluoma,et al.  Explanatory frameworks for interaction design , 2005 .

[102]  Bryant Walker Smith Proximity-Driven Liability , 2013 .

[103]  Andreas Tolk,et al.  8. Modeling Sensing , 2012 .

[104]  Tuncer Ören,et al.  An ontology-based dictionary of understanding as a basis for software agents with understanding abilities , 2007, SpringSim '07.

[105]  David E. Cox,et al.  The Naturalistic Flight Deck System: An Integrated System Concept for Improved Single-Pilot Operations , 2007 .

[106]  G.,et al.  From personal to cultural computing : how to assess a cultural experience , 2006 .

[107]  Raja Parasuraman,et al.  Neuroergonomics: The Brain at Work , 2006 .