Quantilizers: A Safer Alternative to Maximizers for Limited Optimization

In the field of AI, expected utility maximizers are commonly used as a model for idealized agents. However, expected utility maximization can lead to unintended solutions when the utility function does not quantify everything the operators care about: imagine, for example, an expected utility maximizer tasked with winning money on the stock market, which has no regard for whether it accidentally causes a market crash. Once AI systems become sufficiently intelligent and powerful, these unintended solutions could become quite dangerous. In this paper, we describe an alternative to expected utility maximization for powerful AI systems, which we call expected utility quantilization. This could allow the construction of AI systems that do not necessarily fall into strange and unanticipated shortcuts and edge cases in pursuit of their goals.

[1]  Thorsten Joachims,et al.  Counterfactual Risk Minimization: Learning from Logged Bandit Feedback , 2015, ICML.

[2]  Nick Bostrom,et al.  Thinking Inside the Box: Controlling and Using an Oracle AI , 2012, Minds and Machines.

[3]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Benja Fallenstein,et al.  Toward Idealized Decision Theory , 2015, ArXiv.

[5]  C. Goodhart.  Problems of Monetary Management: The UK Experience , 1984 .

[6]  Benja Fallenstein,et al.  Problems of Self-reference in Self-improving Space-Time Embedded Intelligence , 2014, AGI.

[7]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[8]  Chris L. Baker,et al.  Goal Inference as Inverse Planning , 2007 .

[9]  Eric Horvitz,et al.  Problem formulation as the reduction of a decision model , 1990, UAI.

[10]  Daphne Koller,et al.  Making Rational Decisions Using Adaptive Utility Elicitation , 2000, AAAI/IAAI.

[11]  Adeboyejo A. Thompson,et al.  Artificial Evolution in the Physical World , 1997 .

[12]  Nate Soares,et al.  The Value Learning Problem , 2018, Artificial Intelligence Safety and Security.

[13]  C. Robert.  Superintelligence: Paths, Dangers, Strategies , 2017 .

[14]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[15]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[16]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[17]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[18]  Fahiem Bacchus,et al.  Graphical models for preference and utility , 1995, UAI.

[19]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.