Budgeted Learning of Naive-Bayes Classifiers

Frequently, acquiring training data has an associated cost. We consider the situation where the learner may purchase data during training, subject TO a budget. IN particular, we examine the CASE WHERE each feature label has an associated cost, AND the total cost OF ALL feature labels acquired during training must NOT exceed the budget.This paper compares methods FOR choosing which feature label TO purchase next, given the budget AND the CURRENT belief state OF naive Bayes model parameters.Whereas active learning has traditionally focused ON myopic(greedy) strategies FOR query selection, this paper presents a tractable method FOR incorporating knowledge OF the budget INTO the decision making process, which improves performance.

[1]  David G. Stork,et al.  Pattern Classification , 1973 .

[2]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[3]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[6]  Andrew McCallum,et al.  Toward Optimal Active Learning through Sampling Estimation of Error Reduction , 2001, ICML.

[7]  Daphne Koller,et al.  Active Learning for Structure in Bayesian Networks , 2001, IJCAI.

[8]  Michael Lindenbaum,et al.  Selective Sampling for Nearest Neighbor Classifiers , 1999, Machine Learning.

[9]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[10]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[11]  Andrew G. Barto,et al.  Optimal learning: computational procedures for bayes-adaptive markov decision processes , 2002 .

[12]  Dan Roth,et al.  Learning cost-sensitive active classifiers , 2002, Artif. Intell..

[13]  Dana Angluin,et al.  Computational learning theory: survey and selected bibliography , 1992, STOC '92.

[14]  Peter D. Turney.  Types of Cost in Inductive Concept Learning , 2002, ArXiv.

[15]  D. Lizotte,et al.  Budgeted Learning , Part I : The Multi-Armed Bandit Case Omid , 2003 .

[16]  Stuart J. Russell,et al.  Decision Theoretic Subsampling for Induction on Large Databases , 1993, ICML.

[17]  Daphne Koller,et al.  Active Learning for Parameter Estimation in Bayesian Networks , 2000, NIPS.