Machine behaviour

Machines powered by artificial intelligence increasingly mediate our social, cultural, economic and political interactions. Understanding the behaviour of artificial intelligence systems is essential to our ability to control their actions, reap their benefits and minimize their harms. Here we argue that this necessitates a broad scientific research agenda to study machine behaviour that incorporates and expands upon the discipline of computer science and includes insights from across the sciences. We first outline a set of questions that are fundamental to this emerging field and then explore the technical, legal and institutional constraints on the study of machine behaviour.Understanding the behaviour of the machines powered by artificial intelligence that increasingly mediate our social, cultural, economic and political interactions is essential to our ability to control the actions of these intelligent machines, reap their benefits and minimize their harms.

[1]  L. Bromham,et al.  Interdisciplinary research has consistently lower funding success , 2016, Nature.

[2]  Heidi Ledford.  How to solve the world's biggest problems , 2015, Nature.

[3]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[5]  Serge Kernbach,et al.  Re-embodiment of Honeybee Aggregation Behavior in an Artificial Micro-Robotic System , 2009, Adapt. Behav..

[6]  Alex Pentland,et al.  Bots as Virtual Confederates: Design and Ethics , 2016, CSCW.

[7]  Lisanne Bainbridge,et al.  Ironies of automation , 1982, Autom..

[8]  L. Hudson,et al.  Drone Warfare: Blowback from the New American Way of War , 2011 .

[9]  Berkeley J. Dietvorst,et al.  Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err , 2014, Journal of experimental psychology. General.

[10]  Emilio Ferrara,et al.  Social Bots Distort the 2016 US Presidential Election Online Discussion , 2016, First Monday.

[11]  Oren Etzioni,et al.  Artificial intelligence and life in 2030: the one hundred year study on artificial intelligence , 2016 .

[12]  Eric Horvitz,et al.  Discovering Blind Spots of Predictive Models: Representations and Policies for Guided Exploration , 2016, ArXiv.

[13]  Kristina Lerman,et al.  Analyzing Uber's Ride-sharing Economy , 2017, WWW.

[14]  Neil Burch,et al.  Heads-up limit hold’em poker is solved , 2015, Science.

[15]  Dave Cliff,et al.  Too Fast Too Furious - Faster Financial-market Trading Agents Can Give Less Efficient Markets , 2012, ICAART.

[16]  Miriam J. Metzger,et al.  The science of fake news , 2018, Science.

[17]  K. Appel,et al.  Every planar map is four colorable. Part II: Reducibility , 1977 .

[18]  J. Henrich,et al.  The Moral Machine experiment , 2018, Nature.

[19]  Günter J. Hitsch,et al.  Matching and Sorting in Online Dating , 2008 .

[20]  Christo Wilson,et al.  An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace , 2016, WWW.

[21]  Heather Roff,et al.  The Strategic Robot Problem: Lethal Autonomous Weapons in War , 2014 .

[22]  Jun Wang,et al.  Inception Score, Label Smoothing, Gradient Vanishing and -log(D(x)) Alternative , 2017, ArXiv.

[23]  Cynthia Breazeal,et al.  Huggable: The Impact of Embodiment on Promoting Socio-emotional Interactions for Young Pediatric Inpatients , 2018, CHI.

[24]  Amir Globerson,et al.  Nightmare at test time: robust learning by feature deletion , 2006, ICML.

[25]  Alex Pentland,et al.  Social fMRI: Investigating and shaping social mechanisms in the real world , 2011, Pervasive Mob. Comput..

[26]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[27]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[28]  A. F. Adams,et al.  The Survey , 2021, Dyslexia in Higher Education.

[29]  Cynthia Breazeal,et al.  Growing Growth Mindset with a Social Robot Peer , 2017, 2017 12th ACM/IEEE International Conference on Human-Robot Interaction (HRI.

[30]  Andrea Lockerd Thomaz,et al.  Teachable robots: Understanding human teaching behavior to build more effective robot learners , 2008, Artif. Intell..

[31]  Jun Zhao,et al.  'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions , 2018, CHI.

[32]  Shou-De Lin,et al.  Designing the Market Game for a Trading Agent Competition , 2001, IEEE Internet Comput..

[33]  Edwin Olson,et al.  Multipolicy decision-making for autonomous driving via changepoint-based behavior prediction: Theory and experiment , 2017, Auton. Robots.

[34]  Alexander Peysakhovich,et al.  Multi-Agent Cooperation and the Emergence of (Natural) Language , 2016, ICLR.

[35]  Filippo Menczer,et al.  The rise of social bots , 2014, Commun. ACM.

[36]  Sandra Hirche,et al.  Synchrony and Reciprocity: Key Mechanisms for Social Companion Robots in Therapy and Care , 2016, Int. J. Soc. Robotics.

[37]  Paul Voosen,et al.  The AI detectives. , 2017, Science.

[38]  Marc G. Bellemare,et al.  The Arcade Learning Environment: An Evaluation Platform for General Agents (Extended Abstract) , 2012, IJCAI.

[39]  Timnit Gebru,et al.  Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification , 2018, FAT.

[40]  Shane Legg,et al.  Deep Reinforcement Learning from Human Preferences , 2017, NIPS.

[41]  Dan Boneh,et al.  Ensemble Adversarial Training: Attacks and Defenses , 2017, ICLR.

[42]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[43]  Shane Legg,et al.  Psychlab: A Psychology Laboratory for Deep Reinforcement Learning Agents , 2018, ArXiv.

[44]  Avi Feller,et al.  Algorithmic Decision Making and the Cost of Fairness , 2017, KDD.

[45]  Iyad Rahwan,et al.  Closing the AI Knowledge Gap , 2018, ArXiv.

[46]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[47]  Loren G. Terveen,et al.  Exploring the filter bubble: the effect of using recommender systems on content diversity , 2014, WWW.

[48]  Cory Maloney.  Mathematics as a Tool of Manipulation in Modern Society. Review of the book by Cathy O’Neil «Weapons of Math Destruction. How Big Data Increases Inequality and Threatens Democracy» , 2017 .

[49]  Armin Krishnan,et al.  Killer Robots: Legality and Ethicality of Autonomous Weapons , 2009 .

[50]  Margaret E. Roberts.  Censored: Distraction and Diversion Inside China's Great Firewall , 2018 .

[51]  Hany Farid,et al.  The accuracy, fairness, and limits of predicting recidivism , 2018, Science Advances.

[52]  Hod Lipson,et al.  Resilient Machines Through Continuous Self-Modeling , 2006, Science.

[53]  Amos J. Storkey,et al.  Censoring Representations with an Adversary , 2015, ICLR.

[54]  Mu-Chen Chen,et al.  Credit scoring with a data mining approach based on support vector machines , 2007, Expert Syst. Appl..

[55]  Komal Patel.  Testing the Limits of the First Amendment: How a CFAA Prohibition on Online Antidiscrimination Testing Infringes on Protected Speech Activity , 2017 .

[56]  B. Morton.  Fake news. , 2018, Marine pollution bulletin.

[57]  Jon M. Kleinberg,et al.  Mechanisms for (mis)allocating scientific credit , 2011, STOC '11.

[58]  Jing Meng,et al.  Abrupt rise of new machine ecology beyond human response time , 2013, Scientific Reports.

[59]  Lada A. Adamic,et al.  Exposure to ideologically diverse news and opinion on Facebook , 2015, Science.

[60]  Jeffrey T. Hancock,et al.  Experimental evidence of massive-scale emotional contagion through social networks , 2014, Proceedings of the National Academy of Sciences.

[61]  Hiroaki Kitano,et al.  RoboCup: The Robot World Cup Initiative , 1997, AGENTS '97.

[62]  Michael P. Wellman,et al.  Ethical Issues for Autonomous Trading Agents , 2017, Minds and Machines.

[63]  Eric Budish,et al.  The High-Frequency Trading Arms Race: Frequent Batch Auctions as a Market Design Response , 2015 .

[64]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[65]  D. Wegner,et al.  Feeling robots and human zombies: Mind perception and the uncanny valley , 2012, Cognition.

[66]  Carlos Eduardo Scheidegger,et al.  Certifying and Removing Disparate Impact , 2014, KDD.

[67]  A. Wagner.  Robustness and Evolvability in Living Systems , 2005 .

[68]  Chih-Fong Tsai,et al.  Using neural network ensembles for bankruptcy prediction and credit scoring , 2008, Expert Syst. Appl..

[69]  N. Tinbergen.  On aims and methods of Ethology , 2010 .

[70]  Lada A. Adamic,et al.  Computational Social Science , 2009, Science.

[71]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[72]  Stella Boess,et al.  Robot Vacuum Cleaner Personality and Behavior , 2011, Int. J. Soc. Robotics.

[73]  Hae Won Park,et al.  Flat vs. Expressive Storytelling: Young Children’s Learning and Retention of a Social Robot’s Narrative , 2017, Front. Hum. Neurosci..

[74]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[75]  George Bravos,et al.  Understanding Human-Machine Networks: A Cross-Disciplinary Survey , 2015, ArXiv.

[76]  Dayong Wang,et al.  Deep Learning for Identifying Metastatic Breast Cancer , 2016, ArXiv.

[77]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[78]  Michael Kearns,et al.  Reinforcement learning for optimized trade execution , 2006, ICML.

[79]  Meredith Ringel Morris,et al.  Toward Scalable Social Alt Text: Conversational Crowdsourcing as a Tool for Refining Vision-to-Language Technology for the Blind , 2017, HCOMP.

[80]  Eric Horvitz,et al.  Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration , 2016, AAAI.

[81]  Suresh Venkatasubramanian,et al.  Runaway Feedback Loops in Predictive Policing , 2017, FAT.

[82]  Inioluwa Deborah Raji,et al.  Model Cards for Model Reporting , 2018, FAT.

[83]  Radhika Nagpal,et al.  Programmable self-assembly in a thousand-robot swarm , 2014, Science.

[84]  Robin Milner,et al.  A Modal Characterisation of Observable Machine-Behaviour , 1981, CAAP.

[85]  Timnit Gebru,et al.  Datasheets for datasets , 2018, Commun. ACM.

[86]  Vijay Erramilli,et al.  I always feel like somebody's watching me: measuring online behavioural advertising , 2014, CoNEXT.

[87]  Tony Doyle,et al.  Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy , 2017, Inf. Soc..

[88]  Piotr Sapiezynski,et al.  Evidence of complex contagion of information in social media: An experiment using Twitter bots , 2017, PloS one.

[89]  Quanshi Zhang,et al.  Visual interpretability for deep learning: a survey , 2018, Frontiers of Information Technology & Electronic Engineering.

[90]  Andrew M. Rosenfield,et al.  NOISE: How to overcome the high, hidden cost of inconsistent decision making , 2016 .

[91]  Jure Leskovec,et al.  Human Decisions and Machine Predictions , 2017, The quarterly journal of economics.

[92]  R. Johnsen,et al.  Theory and Experiment , 2010 .

[93]  Cynthia Breazeal,et al.  Measuring young children's long-term relationships with social robots , 2018, IDC.

[94]  Roger Bemelmans,et al.  Socially assistive robots in elderly care: a systematic review into effects and effectiveness. , 2012, Journal of the American Medical Directors Association.

[95]  M. Feder.  Robustness and Evolvability in Living Systems. Princeton Studies in Complexity.By Andreas Wagner. Princeton (New Jersey): Princeton University Press. $49.50. xv + 367 p; ill.; index. ISBN: 0–691–12240–7. 2005. , 2006 .

[96]  Tom M. Mitchell,et al.  What can machine learning do? Workforce implications , 2017, Science.

[97]  Kasper Roszbach,et al.  Bank Lending Policy, Credit Scoring, and the Survival of Loans , 2004, Review of Economics and Statistics.

[98]  Sinan Aral,et al.  The spread of true and false news online , 2018, Science.

[99]  K. Appel,et al.  Every Planar Map Is Four Colorable , 2019, Mathematical Solitaires & Games.

[100]  Jon M. Kleinberg,et al.  Inherent Trade-Offs in the Fair Determination of Risk Scores , 2016, ITCS.

[101]  Martin Wattenberg,et al.  SmoothGrad: removing noise by adding noise , 2017, ArXiv.

[102]  K. Appel,et al.  Every planar map is four colorable. Part I: Discharging , 1977 .

[103]  Eric Horvitz,et al.  Combining human and machine intelligence in large-scale crowdsourcing , 2012, AAMAS.

[104]  Nick Feamster,et al.  Take This Personally: Pollution Attacks on Personalized Services , 2013, USENIX Security Symposium.

[105]  Martin Hilbert,et al.  Communicating with Algorithms: A Transfer Entropy Analysis of Emotions-based Escapes from Online Echo Chambers , 2018, Communication Methods and Measures.

[106]  David García,et al.  Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr , 2017, CSCW.

[107]  一樹 美添,et al.  5分で分かる! ? 有名論文ナナメ読み：Silver, D. et al. : Mastering the Game of Go without Human Knowledge , 2018 .

[108]  Fabio Roli,et al.  Evasion Attacks against Machine Learning at Test Time , 2013, ECML/PKDD.

[109]  David Lazer,et al.  The rise of the social algorithm , 2015, Science.

[110]  Adam Tauman Kalai,et al.  Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , 2016, NIPS.

[111]  Andrew T. Hartnett,et al.  This PDF file includes: Materials and Methods SOM Text Figs. S1 to S12 Table S1 Full Reference List , 2022 .

[112]  Michael Carl Tschantz,et al.  Automated Experiments on Ad Privacy Settings , 2014, Proc. Priv. Enhancing Technol..

[113]  W. Kannel,et al.  Diabetes and cardiovascular disease. The Framingham study. , 1979, JAMA.

[114]  Amos Azaria,et al.  The DARPA Twitter Bot Challenge , 2016, Computer.

[115]  M. Meyer,et al.  Two Cheers for Corporate Experimentation: The A/B Illusion and the Virtues of Data-Driven Innovation , 2015 .

[116]  Jonathan Schaeffer,et al.  Checkers Is Solved , 2007, Science.

[117]  Latanya Sweeney,et al.  Discrimination in online ad delivery , 2013, CACM.

[118]  Nicholas A. Christakis,et al.  Locally noisy autonomous agents improve global human coordination in network experiments , 2017, Nature.

[119]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[120]  Spyros Skouras,et al.  An ecological perspective on the future of computer trading , 2013 .

[121]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[122]  A. Pentland,et al.  Social Physics: How Social Networks Can Make Us Smarter , 2014 .

[123]  Randolph M Nesse,et al.  Tinbergen's four questions, organized: a response to Bateson and Laland. , 2013, Trends in ecology & evolution.

[124]  Nicholas R. Jennings,et al.  Human-agent collectives , 2014, CACM.

[125]  I. Couzin,et al.  Emergent Sensing of Complex Environments by Mobile Animal Groups , 2013, Science.

[126]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[127]  Arvind Narayanan,et al.  Semantics derived automatically from language corpora contain human-like biases , 2016, Science.

[128]  Alexandra Chouldechova,et al.  A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions , 2018, FAT.

[129]  Iyad Rahwan,et al.  The social dilemma of autonomous vehicles , 2015, Science.

[130]  Tian-Shyug Lee,et al.  A two-stage hybrid credit scoring model using artificial neural networks and multivariate adaptive regression splines , 2005, Expert Syst. Appl..

[131]  Iyad Rahwan,et al.  Cooperating with machines , 2017, Nature Communications.

[132]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[133]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[134]  A. Menkveld.  The Economics of High-Frequency Trading: Taking Stock , 2016 .

[135]  Michael P. Wellman,et al.  Economic reasoning and artificial intelligence , 2015, Science.

[136]  Hilla Peretz,et al.  The , 1966 .

[137]  Carter C. Price,et al.  Predictive Policing: The Role of Crime Forecasting in Law Enforcement Operations , 2013 .

[138]  J. Sowers,et al.  Diabetes and cardiovascular disease. , 1999, Diabetes care.

[139]  P. Bak,et al.  Self-organized criticality in the 'Game of Life" , 1989, Nature.

[140]  Xiaohua Zeng,et al.  Design and performance evaluation of voice activated wireless home devices , 2006, IEEE Transactions on Consumer Electronics.

[141]  Filiberto Zattoni,et al.  Aims and Methods , 2001, European Urology.

[142]  A. Plasek.  The Social and Economic Implications of Artificial Intelligence Technologies in the Near-Term , 2016 .

[143]  Pedro M. Domingos,et al.  Adversarial classification , 2004, KDD.

[144]  Christo Wilson,et al.  Observing algorithmic marketplaces in-the-wild , 2017, SECO.

[145]  Youyong Kong,et al.  Deep Direct Reinforcement Learning for Financial Signal Representation and Trading , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[146]  Taha Yasseri,et al.  Even good bots fight: The case of Wikipedia , 2016, PloS one.

[147]  Alex Kulesza,et al.  Empirical Limitations on High Frequency Trading Profitability , 2010, 1007.2593.

[148]  Zeynep Tufekci,et al.  Engineering the public: Big data, surveillance and computational politics , 2014, First Monday.

[149]  Rajarshi Das,et al.  Agent-Human Interactions in the Continuous Double Auction , 2001, IJCAI.

[150]  A. Lo,et al.  Moore's Law vs. Murphy's Law: Algorithmic Trading and Its Discontents , 2013 .