Learning with attribute costs

We study an extension of the "standard" learning models to settings where observing the value of an attribute has an associated cost (which might be different for different attributes). Our model assumes that the correct classification is given by some target function f from a class of functions cal F; most of our results discuss the ability to learn a clause (an OR function of a subset of the variables) in various settings:Offline: We are given both the function f and the distribution D that is used to generate an input x. The goal is to design a strategy to decide what attribute of x to observe next so as to minimize the expected evaluation cost of f(x). (In this setting there is no "learning" to be done but only an optimization problem to be solved; this problem to be NP-hard and hence approximation algorithms are presented.)Distributional online: We study two types of "learning" problems; one where the target function f is known to the learner but the distribution D is unknown (and the goal is to minimize the expected cost including the cost that stems from "learning" D), and the other where f is unknown (except that f∈cal F) but D is known (and the goal is to minimize the expected cost while limiting the prediction error involved in "learning" f).Adversarial online: We are given f, however the inputs are selected adversarially. The goal is to compare the learner's cost to that of the best fixed evaluation order (i.e., we analyze the learner's performance by a competitive analysis).

[1]  Leslie G. Valiant,et al.  Fast probabilistic algorithms for hamiltonian circuits and matchings , 1977, STOC '77.

[2]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[3]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[4]  Michael Stonebraker,et al.  Predicate migration: optimizing queries with expensive predicates , 1992, SIGMOD Conference.

[5]  Michael Kharitonov,et al.  Cryptographic hardness of distribution-specific learning , 1993, STOC.

[6]  Mihir Bellare,et al.  On Chromatic Sums and Distributed Resource Allocation , 1998, Inf. Comput..

[7]  Rajeev Motwani,et al.  Randomized algorithms , 1996, CSUR.

[8]  Oren Etzioni,et al.  Efficient information gathering on the Internet , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[9]  Guy Kortsarz,et al.  A Matched Approximation Bound for the Sum of a Greedy Coloring , 1999, Inf. Process. Lett..

[10]  Venkatesan Guruswami,et al.  Query strategies for priced information (extended abstract) , 2000, STOC '00.

[11]  OREN ETZIONI,et al.  Optimal Information Gathering on the Internet with Time and Cost Constraints , 2000, SIAM J. Comput..

[12]  Russell Greiner,et al.  Budgeted learning of nailve-bayes classifiers , 2002, UAI 2002.

[13]  Venkatesan Guruswami,et al.  Query strategies for priced information (extended abstract) , 2000, STOC '00.

[14]  Edith Cohen,et al.  Associative search in peer to peer networks: harnessing latent semantics , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[15]  Russell Greiner,et al.  Budgeted Learning of Naive-Bayes Classifiers , 2003, UAI.

[16]  Santosh S. Vempala,et al.  Efficient algorithms for online decision problems , 2005, J. Comput. Syst. Sci..

[17]  Jennifer Widom,et al.  Adaptive ordering of pipelined stream filters , 2004, SIGMOD '04.

[18]  László Lovász,et al.  Approximating Min Sum Set Cover , 2004, Algorithmica.

[19]  Russell Greiner,et al.  Active Model Selection , 2004, UAI.

[20]  Jennifer Widom,et al.  The Pipelined Set Cover Problem , 2005, ICDT.

[21]  Moshe Tennenholtz,et al.  Overcoming free riding in multi-party computations - The anonymous case , 2006, Games Econ. Behav..