Value of Information Based on Decision Robustness

There are many criteria for measuring the value of information (VOI), each based on a different principle that is usually suitable for specific applications. We propose a new criterion for measuring the value of information, which values information that leads to robust decisions (i.e., ones that are unlikely to change due to new information). We also introduce an algorithm for Naive Bayes networks that selects features with maximal VOI under the new criterion. We discuss the application of the new criterion to classification tasks, showing how it can be used to tradeoff the budget, allotted for acquiring information, with the classification accuracy. In particular, we show empirically that the new criterion can reduce the expended budget significantly while reducing the classification accuracy only slightly. We also show empirically that the new criterion leads to decisions that are much more robust than those based on traditional VOI criteria, such as information gain and classification loss. This make the new criterion particularly suitable for certain decision making applications.

[1]  David Pisinger,et al.  Algorithms for Knapsack Problems , 1995 .

[2]  Adnan Darwiche,et al.  An Exact Algorithm for Computing the Same-Decision Probability , 2013, IJCAI.

[3]  Gladys Castillo,et al.  Using Bayesian networks to improve knowledge assessment , 2013, Comput. Educ..

[4]  Andreas Krause,et al.  Near-optimal Nonmyopic Value of Information in Graphical Models , 2005, UAI.

[5]  Lise Getoor,et al.  Value of Information Lattice: Exploiting Probabilistic Independence for Effective Feature Subset Acquisition , 2011, J. Artif. Intell. Res..

[6]  Jason Stanley,et al.  A Rank-Based Approach to Active Diagnosis , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  A. Darwiche,et al.  Complexity Results and Approximation Strategies for MAP Explanations , 2011, J. Artif. Intell. Res..

[8]  Michael Bögl,et al.  Metaheuristic Search Concepts: A Tutorial with Applications to Production and Logistics , 2010 .

[9]  Andreas Krause,et al.  Adaptive Submodularity: Theory and Applications in Active Learning and Stochastic Optimization , 2010, J. Artif. Intell. Res..

[10]  Deeparnab Chakrabarty,et al.  Knapsack Problems , 2008 .

[11]  Qiang Ji,et al.  Efficient Sensor Selection for Active Information Fusion , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  KURT VANLEHN.  Bayesian student modeling, user interfaces and feedback : A sensitivity analysis , 2001 .

[13]  Richard E. Korf,et al.  Multi-Way Number Partitioning , 2009, IJCAI.

[14]  Tsai-Ching Lu,et al.  Focusing Strategies for Multiple Fault Diagnosis , 2006, FLAIRS.

[15]  Dan Roth,et al.  Learning cost-sensitive active classifiers , 2002, Artif. Intell..

[16]  Daphne Koller,et al.  Active Classification based on Value of Classifier , 2011, NIPS.

[17]  Rómer Rosales,et al.  Active Sensing , 2009, AISTATS.

[18]  Adnan Darwiche,et al.  Same-decision probability: A confidence measure for threshold-based decisions , 2012, Int. J. Approx. Reason..

[19]  Adnan Darwiche,et al.  The Same-Decision Probability: A New Tool for Decision Making , 2012 .

[20]  Yoav Shoham,et al.  Optimal Testing of Structured Knowledge , 2008, AAAI.

[21]  Solomon Eyal Shimony,et al.  Finding MAPs for Belief Networks is NP-Hard , 1994, Artif. Intell..

[22]  Andreas Krause,et al.  Optimal Value of Information in Graphical Models , 2009, J. Artif. Intell. Res..

[23]  Zhendong Niu.  Bayesian student modeling, user interfaces and feedback: A sensitivity analysis , 2001 .

[24]  Adnan Darwiche,et al.  Reasoning about Bayesian Network Classifiers , 2002, UAI.

[25]  S. Martello,et al.  Algorithms for Knapsack Problems , 1987 .

[26]  Cassio Polpo de Campos.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence New Complexity Results for MAP in Bayesian Networks , 2022 .

[27]  Chris H. Q. Ding,et al.  Minimum redundancy feature selection from microarray gene expression data , 2003, Computational Systems Bioinformatics. CSB2003. Proceedings of the 2003 IEEE Bioinformatics Conference. CSB2003.

[28]  Adnan Darwiche,et al.  Algorithms and Applications for the Same-Decision Probability , 2014, J. Artif. Intell. Res..

[29]  Brandi L. Cantarel,et al.  BAYSIC: a Bayesian method for combining sets of genome variants with improved specificity and sensitivity , 2014, BMC Bioinformatics.