A novel multi-task TSK fuzzy classifier and its enhanced version for labeling-risk-aware multi-task classification

While the Takagi-Sugeno-Kang (TSK) fuzzy system has been extensively applied to regression, the aim of this paper is to unveil its potential for classification, of multiple tasks in particular. First, a novel TSK fuzzy classifier (TSK-FC) is presented for pattern classification by integrating the large margin criterion into the objective function. Where multiple tasks are concerned, it has been shown that learning the tasks simultaneously yields better performance than learning them independently. In this regard, the capabilities of TSK-FC are exploited for multi-task learning, through the use of a multi-task TSK fuzzy classifier called MT-TSK-FC. MT-TSK-FC is a mechanism that uses not only the independent sample information of each task, but also the inter-task correlation information to enhance classification performance. However, as the number of tasks increases, the learning process is prone to labeling risk, which can lead to considerable degradation in the performance of pattern classification. To reduce the risk, a labeling-risk-aware mechanism is proposed to enhance the performance of the MT-TSK-FC, thus leading to the development of the labeling-risk-aware multi-task TSK fuzzy classifier called LRA-MT-TSK-FC. Since the three proposed fuzzy classifiers - TSK-FC, MT-TSK-FC, and LRA-MT-TSK-FC - can all be implemented by solving the corresponding QP problems, global optimal solutions are guaranteed. Experiments on multi-task synthetic and real image datasets are conducted to comprehensively demonstrate the effectiveness of the classifiers.

[1]  Shiliang Sun Multitask learning for EEG-based biometrics , 2008, 2008 19th International Conference on Pattern Recognition.

[2]  Shuicheng Yan,et al.  Visual classification with multi-task joint sparse representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Gerardo M. Mendez,et al.  Hybrid learning mechanism for interval A2-C1 type-2 non-singleton type-2 Takagi-Sugeno-Kang fuzzy logic systems , 2013, Inf. Sci..

[4]  Gao Zheng,et al.  A similarity measure between interval type-2 fuzzy sets , 2010, 2010 IEEE International Conference on Mechatronics and Automation.

[5]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[6]  Chia-Feng Juang,et al.  A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms , 2002, IEEE Trans. Fuzzy Syst..

[7]  Oscar Castillo,et al.  A review on type-2 fuzzy logic applications in clustering, classification and pattern recognition , 2014, Appl. Soft Comput..

[8]  Robert Babuska,et al.  Fuzzy Modeling for Control , 1998 .

[9]  Chia-Feng Juang,et al.  TS-fuzzy system-based support vector regression , 2009, Fuzzy Sets Syst..

[10]  Oscar Castillo,et al.  Information granule formation via the concept of uncertainty-based information with Interval Type-2 Fuzzy Sets representation and Takagi-Sugeno-Kang consequents optimized with Cuckoo search , 2015, Appl. Soft Comput..

[11]  M. Balazinski,et al.  Type-2 Takagi-Sugeno-Kang Fuzzy Logic Modeling using Subtractive Clustering , 2006, NAFIPS 2006 - 2006 Annual Meeting of the North American Fuzzy Information Processing Society.

[12]  H. Abdi,et al.  Principal component analysis , 2010 .

[13]  Kilian Q. Weinberger,et al.  Large Margin Multi-Task Metric Learning , 2010, NIPS.

[14]  Jacek M. Leski,et al.  TSK-fuzzy modeling based on /spl epsiv/-insensitive learning , 2005, IEEE Transactions on Fuzzy Systems.

[15]  Zhaohong Deng,et al.  Multitask TSK Fuzzy System Modeling by Mining Intertask Common Hidden Structure , 2015, IEEE Transactions on Cybernetics.

[16]  Oscar Castillo,et al.  Application of interval type-2 fuzzy neural networks in non-linear identification and time series prediction , 2013, Soft Computing.

[17]  Francisco Herrera,et al.  A multi-objective evolutionary method for learning granularities based on fuzzy discretization to improve the accuracy-complexity trade-off of fuzzy rule-based classification systems: D-MOFARC algorithm , 2014, Appl. Soft Comput..

[18]  Zhaohong Deng,et al.  Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods , 2014, IEEE Transactions on Cybernetics.

[19]  Gerardo M. Mendez,et al.  Interval Type-1 Non-Singleton Type-2 TSK Fuzzy Logic Systems Using the Hybrid Training Method RLS-BP , 2007, FOCI.

[20]  Chia-Feng Juang,et al.  Fuzzy System Learned Through Fuzzy Clustering and Support Vector Machine for Human Skin Color Segmentation , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[21]  Shiliang Sun,et al.  Multitask multiclass support vector machines: Model and experiments , 2013, Pattern Recognit..

[22]  Oscar Castillo,et al.  Fuzzy granular gravitational clustering algorithm for multivariate data , 2014, Inf. Sci..

[23]  John Yen,et al.  Improving the interpretability of TSK fuzzy models by combining global learning and local learning , 1998, IEEE Trans. Fuzzy Syst..

[24]  Markus Reischl,et al.  Takagi--Sugeno--Kang Fuzzy Classifiers for a Special Class of Time-Varying Systems , 2008, IEEE Transactions on Fuzzy Systems.

[25]  Hiok Chai Quek,et al.  GA-TSKfnn: Parameters tuning of fuzzy neural network using genetic algorithms , 2005, Expert Syst. Appl..

[26]  Zhaohong Deng,et al.  Scalable TSK Fuzzy Modeling for Very Large Datasets Using Minimal-Enclosing-Ball Approximation , 2011, IEEE Transactions on Fuzzy Systems.

[27]  Ryohei Nakano,et al.  Optimizing Support Vector regression hyperparameters based on cross-validation , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[28]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[29]  Pedro M. Domingos,et al.  Learning Bayesian network classifiers by maximizing conditional likelihood , 2004, ICML.

[30]  Ioannis B. Theocharis,et al.  A GA-based fuzzy modeling approach for generating TSK models , 2002, Fuzzy Sets Syst..

[31]  M. Balazinski,et al.  Joint friction identification for robots using TSK fuzzy system based on subtractive clustering , 2008, NAFIPS 2008 - 2008 Annual Meeting of the North American Fuzzy Information Processing Society.

[32]  George W. Irwin,et al.  A New Gradient Descent Approach for Local Learning of Fuzzy Neural Models , 2013, IEEE Transactions on Fuzzy Systems.

[33]  Yu-Ching Lin,et al.  A TSK-type fuzzy neural network (TFNN) systems for dynamic systems identification , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[34]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[35]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[36]  Korris Fu-Lai Chung,et al.  On minimum distribution discrepancy support vector machine for domain adaptation , 2012, Pattern Recognit..

[37]  Steven C. H. Hoi,et al.  Multiview Semi-Supervised Learning with Consensus , 2012, IEEE Transactions on Knowledge and Data Engineering.

[38]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[39]  Jesús Alcalá-Fdez,et al.  A Fuzzy Association Rule-Based Classification Model for High-Dimensional Problems With Genetic Rule Selection and Lateral Tuning , 2011, IEEE Transactions on Fuzzy Systems.

[40]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[41]  David G. Stork,et al.  Pattern Classification , 1973 .

[42]  Pei-Chann Chang,et al.  A Hybrid System Integrating a Wavelet and TSK Fuzzy Rules for Stock Price Forecasting , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[43]  Song-Shyong Chen,et al.  Robust TSK fuzzy modeling for function approximation with outliers , 2001, IEEE Trans. Fuzzy Syst..

[44]  Zhaohong Deng,et al.  Knowledge-Leverage-Based TSK Fuzzy System Modeling , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[45]  John Q. Gan,et al.  Extracting Takagi-Sugeno Fuzzy Rules with Interpretable Submodels via Regularization of Linguistic Modifiers , 2009, IEEE Transactions on Knowledge and Data Engineering.

[46]  James M. Keller,et al.  Fuzzy Models and Algorithms for Pattern Recognition and Image Processing , 1999 .

[47]  Francisco Herrera,et al.  Enhancing Multiclass Classification in FARC-HD Fuzzy Classifier: On the Synergy Between $n$-Dimensional Overlap Functions and Decomposition Strategies , 2015, IEEE Transactions on Fuzzy Systems.

[48]  Ali Belmehdi,et al.  Multi-objective optimization of TSK fuzzy models , 2008, 2008 5th International Multi-Conference on Systems, Signals and Devices.

[49]  Massimiliano Pontil,et al.  Regularized multi--task learning , 2004, KDD.

[50]  Teck Wee Chua,et al.  Type-2 Fuzzy System for ECG Arrhythmic Classification , 2007, 2007 IEEE International Fuzzy Systems Conference.