Conformal Feature-Selection Wrappers and ensembles for negative-transfer avoidance

Abstract In this paper we propose two methods for instance transfer based on conformal prediction. As a distinctive character, both of the methods are model independent and combine feature selection and source-instance selection to avoid negative transfer. The methods have been tested experimentally for different types of classification model on several benchmark data sets. The experimental results demonstrate that the new methods are capable of outperforming significantly standard instance transfer methods.

[1]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[2]  Hans-Peter Kriegel,et al.  Integrating structured biological data by Kernel Maximum Mean Discrepancy , 2006, ISMB.

[3]  Ralf Peeters,et al.  Conformal decision-tree approach to instance transfer , 2017, Annals of Mathematics and Artificial Intelligence.

[4]  Shotaro Akaho,et al.  TrBagg: A Simple Transfer Learning Method and its Application to Personalization in Collaborative Tagging , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[5]  E. Lehmann,et al.  Nonparametrics: Statistical Methods Based on Ranks , 1976 .

[6]  Paulo Cortez,et al.  Modeling wine preferences by data mining from physicochemical properties , 2009, Decis. Support Syst..

[7]  Ralf Peeters,et al.  Conformity-based source subset selection for instance transfer , 2017, Neurocomputing.

[8]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[9]  Vladimir Vovk.  The Basic Conformal Prediction Framework , 2014 .

[10]  I. Jolliffe.  Principal Component Analysis , 2002 .

[11]  Paulo Cortez,et al.  Using data mining to predict secondary school student performance , 2008 .

[12]  Chandan K. Reddy,et al.  Adaptive Boosting for Transfer Learning Using Dynamic Updates , 2011, ECML/PKDD.

[13]  Qiang Yang,et al.  Transitive Transfer Learning , 2015, KDD.

[14]  Taghi M. Khoshgoftaar,et al.  A survey of transfer learning , 2016, Journal of Big Data.

[15]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[16]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[17]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[18]  Ralf Peeters,et al.  Conformal Region Classification with Instance-Transfer Boosting , 2015, Int. J. Artif. Intell. Tools.

[19]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[20]  Lior Rokach,et al.  Ensemble learning: A survey , 2018, WIREs Data Mining Knowl. Discov..

[21]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[22]  Peter Buser,et al.  BNP-guided vs symptom-guided heart failure therapy: the Trial of Intensified vs Standard Medical Therapy in Elderly Patients With Congestive Heart Failure (TIME-CHF) randomized trial. , 2009, JAMA.

[23]  M. Pfisterer,et al.  Management of elderly patients with congestive heart failure--design of the Trial of Intensified versus standard Medical therapy in Elderly patients with Congestive Heart Failure (TIME-CHF). , 2006, American heart journal.

[24]  Jaime G. Carbonell,et al.  Feature Selection for Transfer Learning , 2011, ECML/PKDD.

[25]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[26]  Ralf Peeters,et al.  Testing exchangeability for transfer decision , 2017, Pattern Recognit. Lett..

[27]  Qiang Yang,et al.  Transferring Naive Bayes Classifiers for Text Classification , 2007, AAAI.

[28]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[29]  D. Aldous.  Exchangeability and related topics , 1985 .

[30]  Jian Zhang,et al.  Double-bootstrapping source data selection for instance-based transfer learning , 2013, Pattern Recognit. Lett..

[31]  Qiang Yang,et al.  Transfer Learning via Dimensionality Reduction , 2008, AAAI.

[32]  Lawrence Carin,et al.  Multi-Task Learning for Classification with Dirichlet Process Priors , 2007, J. Mach. Learn. Res..

[33]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.