Understanding User Attitudes Towards Negative Side Effects of AI Systems

Artificial Intelligence (AI) systems deployed in the open world may produce negative side effects—which are unanticipated, undesirable outcomes that occur in addition to the intended outcomes of the system’s actions. These negative side effects affect users directly or indirectly, by violating their preferences or altering their environment in an undesirable, potentially harmful, manner. While the existing literature has started to explore techniques to overcome the impacts of negative side effects in deployed systems, there has been no prior efforts to determine how users perceive and respond to negative side effects. We surveyed 183 participants to develop an understanding of user attitudes towards side effects and how side effects impact user trust in the system. The surveys targeted two domains: an autonomous vacuum cleaner and an autonomous vehicle, each with 183 respondents. The results indicate that users are willing to tolerate side effects that are not safety-critical but prefer to minimize them as much as possible. Furthermore, users are willing to assist the system in mitigating negative side effects by providing feedback and reconfiguring the environment. Trust in the system diminishes if it fails to minimize the impacts of negative side effects over time. These results support key fundamental assumptions in existing techniques and facilitate the development of new methods to overcome negative side effects of AI systems.

[1]  Shlomo Zilberstein,et al.  Mitigating Negative Side Effects via Environment Shaping , 2021, AAMAS.

[2]  Shlomo Zilberstein,et al.  Avoiding Negative Side Effects due to Incomplete Knowledge of AI Systems , 2020, AI Mag..

[3]  Shlomo Zilberstein,et al.  A Multi-Objective Approach to Mitigate Negative Side Effects , 2020, IJCAI.

[4]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[5]  Moritz Körber,et al.  Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation , 2018, Advances in Intelligent Systems and Computing.

[6]  Edmund H. Durfee,et al.  Minimax-Regret Querying on Side Effects for Safe Optimality in Factored Markov Decision Processes , 2018, IJCAI.

[7]  Anca D. Dragan,et al.  Inverse Reward Design , 2017, NIPS.

[8]  Laurent Orseau,et al.  Penalizing Side Effects using Stepwise Relative Reachability , 2018, AISafety@IJCAI.

[9]  Riender Happee,et al.  Public opinion on automated driving: results of an international questionnaire among 5000 respondents , 2015 .

[10]  Anca D. Dragan,et al.  Preferences Implicit in the State of the World , 2018, ICLR.

[11]  Paul N. Bennett,et al.  Will You Accept an Imperfect AI?: Exploring Designs for Adjusting End-user Expectations of AI Systems , 2019, CHI.

[12]  Bill Hibbard,et al.  Avoiding Unintended AI Behaviors , 2012, AGI.

[13]  Berkeley J. Dietvorst,et al.  Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err , 2014, Journal of experimental psychology. General.

[14]  Jodi Forlizzi,et al.  Service robots in the domestic environment: a study of the roomba vacuum in the home , 2006, HRI '06.

[15]  Ming Yin,et al.  Understanding the Effect of Accuracy on Trust in Machine Learning Models , 2019, CHI.

[16]  Edwin R. Galea,et al.  Perceptions of autonomous vehicles: Relationships with road users, risk, gender and age , 2018 .

[17]  Joseph P. Simmons,et al.  Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them , 2016, Manag. Sci..

[18]  Lorrie Faith Cranor,et al.  Are your participants gaming the system?: screening mechanical turk workers , 2010, CHI.

[19]  Laurent Orseau,et al.  Avoiding Side Effects By Considering Future Tasks , 2020, NeurIPS.

[20]  Dylan Hadfield-Menell,et al.  Conservative Agency via Attainable Utility Preservation , 2019, AIES.

[21]  Eric Horvitz,et al.  Discovering Blind Spots in Reinforcement Learning , 2018, AAMAS.