A Clustering based Selection Framework for Cost Aware and Test-time Feature Elicitation

Most learning algorithms are optimized with generalization and predictive performance as the goal. However, in most real-world machine learning applications, obtaining features at test time can incur a cost. For example, in clinical tasks, acquiring certain features such as FMRI or certain lab tests for patients can be expensive, while other features like patient demography or history are easily obtained and do not have a cost involved. Motivated by this, we address the problem of test-time elicitation of features. We formulate the problem of cost-aware feature elicitation as an optimization problem with trade-off between performance and feature acquisition cost. We assume that the cost of the features has already been paid in obtaining the training data. We propose a Clustering based Cost Aware Test-time Feature Elicitation (CATE) algorithm, which can select the relevant feature set given the observed attributes of the test instance. Our experiments on four real-world tasks demonstrate the efficacy and effectiveness of our proposed approach in both cost and performance.

[1]  Venkatesh Saligrama,et al.  Pruning Random Forests for Prediction on a Budget , 2016, NIPS.

[2]  Christian Osendorfer,et al.  Sequential Feature Selection for Classification , 2011, Australasian Conference on Artificial Intelligence.

[3]  Pallika H. Kanani.  Prediction-time Active Feature-value Acquisition for Cost-effective Customer Targeting , 2008 .

[4]  Sriraam Natarajan,et al.  Identifying Parkinson's Patients: A Functional Gradient Boosting Approach , 2017, AIME.

[5]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection , 2012, J. Mach. Learn. Res..

[6]  Daphne Koller,et al.  Active Classification based on Value of Classifier , 2011, NIPS.

[7]  Venkatesh Saligrama,et al.  Fast margin-based cost-sensitive classification , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Foster J. Provost,et al.  Active Feature-Value Acquisition , 2009, Manag. Sci..

[9]  Kilian Q. Weinberger,et al.  The Greedy Miser: Learning under Test-time Budgets , 2012, ICML.

[10]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[11]  Foster J. Provost,et al.  An expected utility approach to active feature-value acquisition , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[12]  Russell Greiner,et al.  Budgeted Learning of Naive-Bayes Classifiers , 2003, UAI.

[13]  Qiang Yang,et al.  Decision trees with minimal costs , 2004, ICML.

[14]  Bálint Antal,et al.  An ensemble-based system for automatic screening of diabetic retinopathy , 2014, Knowl. Based Syst..

[15]  Victor S. Sheng,et al.  Feature value acquisition in testing: a sequential batch test algorithm , 2006, ICML.

[16]  Zhiqiang Zheng,et al.  On active learning for data acquisition , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[17]  Qiang Yang,et al.  Test-cost sensitive naive Bayes classification , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[18]  Eunho Yang,et al.  Joint Active Feature Acquisition and Classification with Variable-Size Set Encoding , 2018, NeurIPS.

[19]  Venkatesh Saligrama,et al.  Feature-Budgeted Random Forest , 2015, ICML.

[20]  Sebastian Nowozin,et al.  EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE , 2018, ICML.

[21]  Ludovic Denoyer,et al.  Datum-Wise Classification: A Sequential Approach to Sparsity , 2011, ECML/PKDD.

[22]  Venkatesh Saligrama,et al.  Efficient Learning by Directed Acyclic Graph For Resource Constrained Prediction , 2015, NIPS.

[23]  Venkatesh Saligrama,et al.  Adaptive Classification for Prediction Under a Budget , 2017, NIPS.

[24]  Hui Lin,et al.  Multi-document Summarization via Budgeted Maximization of Submodular Functions , 2010, NAACL.

[25]  Rishabh K. Iyer,et al.  Submodular Optimization and Machine Learning: Theoretical Results, Unifying and Scalable Algorithms, and Applications , 2015 .

[26]  Shuo Yang,et al.  Identifying Rare Diseases from Behavioural Data: A Machine Learning Approach , 2016, 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE).

[27]  A. Singleton,et al.  The Parkinson Progression Marker Initiative (PPMI) , 2011, Progress in Neurobiology.

[28]  Foster J. Provost,et al.  Active feature-value acquisition for classifier induction , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[29]  Sriraam Natarajan,et al.  On Whom Should I Perform this Lab Test Next? An Active Feature Elicitation Approach , 2018, IJCAI.