Combination of Transferable Classification With Multisource Domain Adaptation Based on Evidential Reasoning

In applications of domain adaptation, there may exist multiple source domains, which can provide more or less complementary knowledge for pattern classification in the target domain. In order to improve the classification accuracy, a decision-level combination method is proposed for the multisource domain adaptation based on evidential reasoning. The classification results obtained from different source domains usually have different reliabilities/weights, which are calculated according to domain consistency. Therefore, the multiple classification results are discounted by the corresponding weights under belief functions framework, and then, Dempster's rule is employed to combine these discounted results. In order to reduce errors, a neighborhood-based cautious decision-making rule is developed to make the class decision depending on the combination result. The object is assigned to a singleton class if its neighborhoods can be (almost) correctly classified. Otherwise, it is cautiously committed to the disjunction of several possible classes. By doing this, we can well characterize the partial imprecision of classification and reduce the error risk as well. A unified utility value is defined here to reflect the benefit of such classification. This cautious decision-making rule can achieve the maximum unified utility value because partial imprecision is considered better than an error. Several real data sets are used to test the performance of the proposed method, and the experimental results show that our new method can efficiently improve the classification accuracy with respect to other related combination methods.

[1]  Ivor W. Tsang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Domain Adaptation from Multiple Sources: A Domain- , 2022 .

[2]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[3]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[4]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[5]  Shiliang Sun,et al.  A survey of multi-source domain adaptation , 2015, Inf. Fusion.

[6]  Thierry Denoeux,et al.  Analysis of evidence-theoretic decision rules for pattern classification , 1997, Pattern Recognit..

[7]  M. Beynon,et al.  The Dempster-Shafer theory of evidence: an alternative approach to multicriteria decision modelling , 2000 .

[8]  Ming Shao,et al.  Incomplete Multisource Transfer Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Jing Zhang,et al.  Joint Geometrical and Statistical Alignment for Visual Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Ling Shao,et al.  Transfer Learning for Visual Categorization: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Thierry Denoeux,et al.  A k-nearest neighbor classification rule based on Dempster-Shafer theory , 1995, IEEE Trans. Syst. Man Cybern..

[12]  Lorenzo Bruzzone,et al.  Semisupervised Transfer Component Analysis for Domain Adaptation in Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Thierry Denoeux,et al.  Decision-Making with Belief Functions: a Review , 2018, Int. J. Approx. Reason..

[14]  Jian-Bo Yang,et al.  On the evidential reasoning algorithm for multiple attribute decision analysis under uncertainty , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[15]  Philippe Smets,et al.  Decision making in the TBM: the necessity of the pignistic transformation , 2005, Int. J. Approx. Reason..

[16]  Denis Deratani Mauá Equivalences between maximum a posteriori inference in Bayesian networks and maximum expected utility computation in influence diagrams , 2016, Int. J. Approx. Reason..

[17]  Rama Chellappa,et al.  Unsupervised Adaptation Across Domain Shifts by Generating Intermediate Data Representations , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Qiang Yang,et al.  Heterogeneous Transfer Learning for Image Classification , 2011, AAAI.

[19]  Thierry Denoeux,et al.  Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective , 2018, Knowl. Based Syst..

[20]  Philip S. Yu,et al.  Visual Domain Adaptation with Manifold Embedded Distribution Alignment , 2018, ACM Multimedia.

[21]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[22]  Shanlin Yang,et al.  Data-driven group decision making for diagnosis of thyroid nodule , 2019, Science China Information Sciences.

[23]  Quan Pan,et al.  Combination of Classifiers With Optimal Weight Based on Evidential Reasoning , 2018, IEEE Transactions on Fuzzy Systems.

[24]  Arthur P. Dempster,et al.  Upper and Lower Probabilities Induced by a Multivalued Mapping , 1967, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[25]  Hui Xiong,et al.  A Unified Framework for Metric Transfer Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[26]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[27]  Koby Crammer,et al.  Analysis of Representations for Domain Adaptation , 2006, NIPS.

[28]  Thierry Denoeux,et al.  A neural network classifier based on Dempster-Shafer theory , 2000, IEEE Trans. Syst. Man Cybern. Part A.

[29]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[30]  Kate Saenko,et al.  Subspace Distribution Alignment for Unsupervised Domain Adaptation , 2015, BMVC.

[31]  Kate Saenko,et al.  Return of Frustratingly Easy Domain Adaptation , 2015, AAAI.

[32]  Jafar Tahmoresnezhad,et al.  Visual domain adaptation via transfer feature learning , 2017, Knowledge and Information Systems.

[33]  Quan Pan,et al.  Classifier Fusion With Contextual Reliability Evaluation , 2018, IEEE Transactions on Cybernetics.

[34]  Denis Deratani Mauá,et al.  Evaluating credal classifiers by utility-discounted predictive accuracy , 2012, Int. J. Approx. Reason..

[35]  Tetsuya Murai,et al.  Multiple-attribute decision making under uncertainty: the evidential reasoning approach revisited , 2006, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[36]  Jean Dezert,et al.  Credal c-means clustering method based on belief functions , 2015, Knowl. Based Syst..

[37]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[38]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Philip S. Yu,et al.  Stratified Transfer Learning for Cross-domain Activity Recognition , 2017, 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[40]  Jean Dezert,et al.  Evidential Reasoning for Multi-Criteria Analysis Based on DSmT-AHP , 2011 .

[41]  Jianmin Wang,et al.  Unsupervised Domain Adaptation With Distribution Matching Machines , 2018, AAAI.

[42]  Tinne Tuytelaars,et al.  Unsupervised Visual Domain Adaptation Using Subspace Alignment , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Francisco Herrera,et al.  Hesitant Fuzzy Linguistic Term Sets for Decision Making , 2012, IEEE Transactions on Fuzzy Systems.

[44]  Juan José del Coz,et al.  Learning Nondeterministic Classifiers , 2009, J. Mach. Learn. Res..

[45]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[46]  Philip S. Yu,et al.  Transfer Joint Matching for Unsupervised Domain Adaptation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Yi Yao,et al.  Boosting for transfer learning with multiple sources , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[48]  Koby Crammer,et al.  A theory of learning from different domains , 2010, Machine Learning.

[49]  Yiqiang Chen,et al.  Balanced Distribution Adaptation for Transfer Learning , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[50]  Gang Wang,et al.  Boosting for transfer learning from multiple data sources , 2012, Pattern Recognit. Lett..

[51]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[52]  Thierry Denoeux,et al.  Making Set-Valued Predictions in Evidential Classification: A Comparison of Different Approaches , 2019, ISIPTA.

[53]  Yu Liu,et al.  Evidence Combination Based on Credal Belief Redistribution for Pattern Classification , 2020, IEEE Transactions on Fuzzy Systems.