Submodular Surrogates for Value of Information

How should we gather information to make effective decisions? A classical answer to this fundamental problem is given by the decision-theoretic value of information. Unfortunately, optimizing this objective is intractable, and myopic (greedy) approximations are known to perform poorly. In this paper, we introduce DIRECT, an efficient yet near-optimal algorithm for nonmyopically optimizing value of information. Crucially, DIRECT uses a novel surrogate objective that is: (1) aligned with the value of information problem (2) efficient to evaluate and (3) adaptive submod-ular. This latter property enables us to utilize an efficient greedy optimization while providing strong approximation guarantees. We demonstrate the utility of our approach on four diverse case-studies: touch-based robotic localization, comparison-based preference learning, wild-life conservation management, and preference elicitation in behavioral economics. In the first application, we demonstrate DIRECT in closed-loop on an actual robotic platform.

[1]  W. Sharpe.  CAPITAL ASSET PRICES: A THEORY OF MARKET EQUILIBRIUM UNDER CONDITIONS OF RISK* , 1964 .

[2]  Ronald A. Howard,et al.  Information Value Theory , 1966, IEEE Trans. Syst. Sci. Cybern..

[3]  D. J. A. Welsh,et al.  An upper bound for the chromatic number of a graph and its application to timetabling problems , 1967, Comput. J..

[4]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[5]  Peter M. Will,et al.  An Experimental System for Computer Controlled Mechanical Assembly , 1975, IEEE Transactions on Computers.

[6]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[7]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[8]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[9]  John Riedl,et al.  An algorithmic framework for performing collaborative filtering , 1999, SIGIR '99.

[10]  Teresa M. Przytycka,et al.  On an Optimal Split Tree Problem , 1999, WADS.

[11]  Igor Kononenko,et al.  Machine learning for medical diagnosis: history, state of the art and perspective , 2001, Artif. Intell. Medicine.

[12]  Sanjoy Dasgupta,et al.  Analysis of a greedy active learning strategy , 2004, NIPS.

[13]  Joelle Pineau,et al.  Anytime Point-Based Approximations for Large POMDPs , 2006, J. Artif. Intell. Res..

[14]  W. Sharpe,et al.  Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk , 2007 .

[15]  Mukesh K. Mohania,et al.  Decision trees for entity identification: approximation algorithms and hardness results , 2007, PODS '07.

[16]  Steve Hanneke,et al.  A bound on the label complexity of agnostic active learning , 2007, ICML '07.

[17]  John Langford,et al.  Agnostic active learning , 2006, J. Comput. Syst. Sci..

[18]  Andreas Krause,et al.  Optimal Value of Information in Graphical Models , 2009, J. Artif. Intell. Res..

[19]  David Cohn,et al.  Active Learning , 2010, Encyclopedia of Machine Learning.

[20]  Andreas Krause,et al.  Near-Optimal Bayesian Active Learning with Noisy Observations , 2010, NIPS.

[21]  P. Wakker.  Prospect Theory: For Risk and Ambiguity , 2010 .

[22]  Jeff A. Bilmes,et al.  Simultaneous Learning and Covering with Adversarial Noise , 2011, ICML.

[23]  Amin Karbasi,et al.  Content Search through Comparisons , 2011, ICALP.

[24]  Andreas Krause,et al.  Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization , 2010, J. Artif. Intell. Res..

[25]  Sarah J. Converse,et al.  Special Issue Article: Adaptive management for biodiversity conservation in an uncertain world Which uncertainty? Using expert elicitation and expected value of information to design an adaptive program , 2011 .

[26]  Adnan Darwiche,et al.  Same-decision probability: A confidence measure for threshold-based decisions , 2012, Int. J. Approx. Reason..

[27]  Andreas Krause,et al.  Bayesian Rapid Optimal Adaptive Design (BROAD): Method and application distinguishing models of risky choice , 2019 .

[28]  Amin Karbasi,et al.  Comparison-Based Learning with Rank Nets , 2012, ICML.

[29]  Siddhartha S. Srinivasa,et al.  Efficient touch based localization through submodularity , 2012, 2013 IEEE International Conference on Robotics and Automation.

[30]  Andreas Krause,et al.  Near Optimal Bayesian Active Learning for Decision Making , 2014, AISTATS.

[31]  Lisa Hellerstein,et al.  Approximation Algorithms for Stochastic Boolean Function Evaluation and Stochastic Submodular Set Cover , 2013, SODA.

[32]  Adnan Darwiche,et al.  Algorithms and Applications for the Same-Decision Probability , 2014, J. Artif. Intell. Res..