A multiphase cost-sensitive learning method based on the multiclass three-way decision-theoretic rough set model

Abstract The decision-theoretic rough set (DTRS) can be regarded as a type of cost-sensitive learning method that incorporates cost functions. Three-way decisions for an object are made in the DTRS model to obtain the minimum Bayesian decision cost, including acceptance, rejection and deferment decisions. However, because the cost matrix in traditional cost-sensitive learning problems does not include the cost for deferment decisions, DTRS cannot be directly used to solve traditional cost-sensitive learning problems. In this paper, we present a multiclass three-way decision-theoretic rough set model that can be directly applied to traditional cost-sensitive learning problems. This model can calculate the corresponding cost for deferment decisions based on the traditional cost matrix. In addition, we propose a multiphase cost-sensitive learning method based on the multiclass three-way decision-theoretic rough set model. The deferment and rejection samples are reduced and eliminated through multiphase iterative learning. Compared with the existing multiclass cost-sensitive learning methods, our proposed method can achieve higher classification accuracy and lower misclassification cost.

[1]  Xiuyi Jia,et al.  Two-Phase Classification Based on Three-Way Decisions , 2013, RSKT.

[2]  Yiyu Yao,et al.  Three-way decisions with probabilistic rough sets , 2010, Inf. Sci..

[3]  M. Maloof Learning When Data Sets are Imbalanced and When Costs are Unequal and Unknown , 2003 .

[4]  Decui Liang,et al.  A Novel Risk Decision Making Based on Decision-Theoretic Rough Sets Under Hesitant Fuzzy Information , 2015, IEEE Transactions on Fuzzy Systems.

[5]  Xinye Cai,et al.  Neighborhood based decision-theoretic rough set models , 2016, Int. J. Approx. Reason..

[6]  Fan Li,et al.  An extension to Rough c-means clustering based on decision-theoretic Rough Sets model , 2014, Int. J. Approx. Reason..

[7]  Weihua Xu,et al.  Generalized multigranulation double-quantitative decision-theoretic rough set , 2016, Knowl. Based Syst..

[8]  Guoyin Wang,et al.  A tree-based incremental overlapping clustering method using the three-way decision theory , 2016, Knowl. Based Syst..

[9]  Lu Wang,et al.  Multi-objective attribute reduction in three-way decision-theoretic rough set model , 2019, Int. J. Approx. Reason..

[10]  Wei-Zhi Wu,et al.  Inclusion measure-based multi-granulation decision-theoretic rough sets in multi-scale intuitionistic fuzzy information tables , 2020, Inf. Sci..

[11]  Guoyin Wang,et al.  An active three-way clustering method via low-rank matrices for multi-view data , 2020, Inf. Sci..

[12]  Decui Liang,et al.  Fuzzy interval decision-theoretic rough sets , 2013, 2013 Joint IFSA World Congress and NAFIPS Annual Meeting (IFSA/NAFIPS).

[13]  Jiye Liang,et al.  International Journal of Approximate Reasoning Multigranulation Decision-theoretic Rough Sets , 2022 .

[14]  Tianrui Li,et al.  THREE-WAY GOVERNMENT DECISION ANALYSIS WITH DECISION-THEORETIC ROUGH SETS , 2012 .

[15]  Nouman Azam,et al.  Web-Based Medical Decision Support Systems for Three-Way Medical Decision Making With Game-Theoretic Rough Sets , 2015, IEEE Transactions on Fuzzy Systems.

[16]  Zhenmin Tang,et al.  Minimum cost attribute reduction in decision-theoretic rough set models , 2013, Inf. Sci..

[17]  Guoyin Wang,et al.  A Decision-Theoretic Rough Set Approach for Dynamic Data Mining , 2015, IEEE Transactions on Fuzzy Systems.

[18]  John Langford,et al.  An iterative method for multi-class cost-sensitive learning , 2004, KDD.

[19]  Yiyu Yao,et al.  Three-way Investment Decisions with Decision-theoretic Rough Sets , 2011, Int. J. Comput. Intell. Syst..

[20]  Decui Liang,et al.  Incorporating logistic regression to decision-theoretic rough sets for classifications , 2014, Int. J. Approx. Reason..

[21]  Hsuan-Tien Lin Reduction from Cost-Sensitive Multiclass Classification to One-versus-One Binary Classification , 2014, ACML.

[22]  Zhenmin Tang,et al.  On an optimization representation of decision-theoretic rough set model , 2014, Int. J. Approx. Reason..

[23]  Yiyu Yao,et al.  Attribute reduction in decision-theoretic rough set models , 2008, Inf. Sci..

[24]  Bao Qing Hu,et al.  Three-way decisions with decision-theoretic rough sets in multiset-valued information tables , 2020, Inf. Sci..

[25]  Hamido Fujita,et al.  Fuzzy neighborhood covering for three-way classification , 2020, Inf. Sci..

[26]  Yiyu Yao,et al.  A Decision Theoretic Framework for Approximating Concepts , 1992, Int. J. Man Mach. Stud..

[27]  Robert C. Holte,et al.  C4.5, Class Imbalance, and Cost Sensitivity: Why Under-Sampling beats Over-Sampling , 2003 .

[28]  Jerzy W. Grzymala-Busse,et al.  Rough Sets , 1995, Commun. ACM.

[29]  Kai Ming Ting,et al.  An Instance-weighting Method to Induce Cost-sensitive Trees , 2001 .

[30]  Charles Elkan,et al.  The Foundations of Cost-Sensitive Learning , 2001, IJCAI.

[31]  Pedro M. Domingos MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[32]  Bartosz Krawczyk Cost-sensitive one-vs-one ensemble for multi-class imbalanced data , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[33]  Dun Liu,et al.  A Multiple-category Classification Approach with Decision-theoretic Rough Sets , 2012, Fundam. Informaticae.

[34]  Witold Pedrycz,et al.  Determining Three-Way Decisions With Decision-Theoretic Rough Sets Using a Relative Value Approach , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[35]  Maya R. Gupta,et al.  Cost-sensitive multi-class classification from probability estimates , 2008, ICML '08.

[36]  Min Chen,et al.  Rough Multi-category Decision Theoretic Framework , 2008, RSKT.

[37]  Xiuyi Jia,et al.  Three-Way Decisions Versus Two-Way Decisions on Filtering Spam Email , 2014, Trans. Rough Sets.

[38]  Zhi-Hua Zhou,et al.  ON MULTI‐CLASS COST‐SENSITIVE LEARNING , 2006, Comput. Intell..

[39]  Yue Xu,et al.  Dual pattern-enhanced representations model for query-focused multi-document summarisation , 2019, Knowl. Based Syst..

[40]  Raymond Y. K. Lau,et al.  Enhancing Binary Classification by Modeling Uncertain Boundary in Three-Way Decisions , 2017, IEEE Transactions on Knowledge and Data Engineering.

[41]  Fan Min,et al.  Tri-partition cost-sensitive active learning through kNN , 2017, Soft Computing.

[42]  Witold Pedrycz,et al.  International Journal of Approximate Reasoning Triangular Fuzzy Decision-theoretic Rough Sets , 2022 .

[43]  Gang Wang,et al.  Research on Cost-Sensitive Method for Boundary Region in Three-Way Decision Model , 2016, IJCRS.

[44]  Bing Huang,et al.  Cost-Sensitive Three-Way Decision: A Sequential Strategy , 2013, RSKT.

[45]  Bing Zhou,et al.  Multi-class decision-theoretic rough sets , 2014, Int. J. Approx. Reason..

[46]  Yang Wang,et al.  Boosting for Learning Multiple Classes with Imbalanced Class Distribution , 2006, Sixth International Conference on Data Mining (ICDM'06).

[47]  Xin Yang,et al.  A unified framework of dynamic three-way probabilistic rough sets , 2017, Inf. Sci..

[48]  Bing Huang,et al.  Sequential three-way decision and granulation for cost-sensitive face recognition , 2016, Knowl. Based Syst..

[49]  Zeshui Xu,et al.  A New Aggregation Method-Based Error Analysis for Decision-Theoretic Rough Sets and Its Application in Hesitant Fuzzy Information Systems , 2017, IEEE Transactions on Fuzzy Systems.

[50]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.