Enhanced Knowledge-Leverage-Based TSK Fuzzy System Modeling for Inductive Transfer Learning

The knowledge-leverage-based Takagi--Sugeno--Kang fuzzy system (KL-TSK-FS) modeling method has shown promising performance for fuzzy modeling tasks where transfer learning is required. However, the knowledge-leverage mechanism of the KL-TSK-FS can be further improved. This is because available training data in the target domain are not utilized for the learning of antecedents and the knowledge transfer mechanism from a source domain to the target domain is still too simple for the learning of consequents when a Takagi--Sugeno--Kang fuzzy system (TSK-FS) model is trained in the target domain. The proposed method, that is, the enhanced KL-TSK-FS (EKL-TSK-FS), has two knowledge-leverage strategies for enhancing the parameter learning of the TSK-FS model for the target domain using available information from the source domain. One strategy is used for the learning of antecedent parameters, while the other is for consequent parameters. It is demonstrated that the proposed EKL-TSK-FS has higher transfer learning abilities than the KL-TSK-FS. In addition, the EKL-TSK-FS has been further extended for the scene of the multisource domain.

[1]  Zhaohong Deng,et al.  Scalable TSK Fuzzy Modeling for Very Large Datasets Using Minimal-Enclosing-Ball Approximation , 2011, IEEE Transactions on Fuzzy Systems.

[2]  Raymond J. Mooney,et al.  Transfer Learning by Mapping with Minimal Target Data , 2008 .

[3]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[5]  Zhaohong Deng,et al.  Transductive domain adaptive learning for epileptic electroencephalogram recognition , 2014, Artif. Intell. Medicine.

[6]  Fuzhen Zhuang,et al.  Triplex Transfer Learning: Exploiting Both Shared and Distinct Concepts for Text Classification , 2013, IEEE Transactions on Cybernetics.

[7]  Korris Fu-Lai Chung,et al.  Transfer Spectral Clustering , 2012, ECML/PKDD.

[8]  Qiang Yang,et al.  Self-taught clustering , 2008, ICML '08.

[9]  Raymond J. Mooney,et al.  Mapping and Revising Markov Logic Networks for Transfer Learning , 2007, AAAI.

[10]  J. Mendel.  Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions , 2001 .

[11]  Jacek M. Leski,et al.  TSK-fuzzy modeling based on /spl epsiv/-insensitive learning , 2005, IEEE Transactions on Fuzzy Systems.

[12]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[13]  Zhaohong Deng,et al.  Knowledge-Leverage-Based Fuzzy System and Its Modeling , 2013, IEEE Transactions on Fuzzy Systems.

[14]  Zhaohong Deng,et al.  Enhanced soft subspace clustering integrating within-cluster and between-cluster information , 2010, Pattern Recognit..

[15]  Longbing Cao,et al.  Knowledge-leverage based TSK fuzzy system with improved knowledge transfer , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[16]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[17]  Anton Schwaighofer,et al.  Learning Gaussian Process Kernels via Hierarchical Bayes , 2004, NIPS.

[18]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[19]  Lawrence Carin,et al.  Logistic regression with an auxiliary data source , 2005, ICML.

[20]  Ivor W. Tsang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Domain Adaptation from Multiple Sources: A Domain- , 2022 .

[21]  Ling Shao,et al.  Transfer Learning for Visual Categorization: A Survey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Steffen Bickel,et al.  Discriminative learning for differing training and test distributions , 2007, ICML '07.

[23]  Fuzhen Zhuang,et al.  Supervised Representation Learning: Transfer Learning with Deep Autoencoders , 2015, IJCAI.

[24]  Qiang Yang,et al.  Boosting for transfer learning , 2007, ICML '07.

[25]  Jieping Ye,et al.  Deep Model Based Transfer and Multi-Task Learning for Biological Image Analysis , 2015, IEEE Transactions on Big Data.

[26]  Peng Hao,et al.  Transfer learning using computational intelligence: A survey , 2015, Knowl. Based Syst..

[27]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[28]  Korris Fu-Lai Chung,et al.  On minimum distribution discrepancy support vector machine for domain adaptation , 2012, Pattern Recognit..

[29]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[30]  Jacek M. Zurada,et al.  Generalized Core Vector Machines , 2006, IEEE Transactions on Neural Networks.

[31]  E. H. Mandami.  Application of Fuzzy Logic to Approximate Reasoning using Linguistic Synthesis , 1977 .

[32]  Hai Yang,et al.  ACM Transactions on Intelligent Systems and Technology - Special Section on Urban Computing , 2014 .

[33]  Chia-Feng Juang,et al.  Fuzzy System Learned Through Fuzzy Clustering and Support Vector Machine for Human Skin Color Segmentation , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[34]  Barbara Caputo,et al.  Learning Categories From Few Examples With Multi Model Knowledge Transfer , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Barbara Caputo,et al.  Learning to Learn, from Transfer Learning to Domain Adaptation: A Unifying Perspective , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[37]  Chia-Feng Juang,et al.  TS-fuzzy system-based support vector regression , 2009, Fuzzy Sets Syst..

[38]  Jun Huan,et al.  Large margin transductive transfer learning , 2009, CIKM.

[39]  Neil D. Lawrence,et al.  Learning to learn with the informative vector machine , 2004, ICML.

[40]  Michael K. Ng,et al.  An Entropy Weighting k-Means Algorithm for Subspace Clustering of High-Dimensional Sparse Data , 2007, IEEE Transactions on Knowledge and Data Engineering.

[41]  Changshui Zhang,et al.  Transferred Dimensionality Reduction , 2008, ECML/PKDD.

[42]  Pedro M. Domingos,et al.  Deep transfer via second-order Markov logic , 2009, ICML '09.

[43]  Qiang Yang,et al.  Transfer learning in heterogeneous collaborative filtering domains , 2013, Artif. Intell..

[44]  Pei Yang,et al.  Bayesian Task-Level Transfer Learning for Non-linear Regression , 2008, 2008 International Conference on Computer Science and Software Engineering.

[45]  Bernhard Schölkopf,et al.  Correcting Sample Selection Bias by Unlabeled Data , 2006, NIPS.

[46]  Madasu Hanmandlu,et al.  Generalization of adaptive neuro-fuzzy inference systems , 2000, IEEE Trans. Neural Networks Learn. Syst..

[47]  Philip S. Yu,et al.  Domain Invariant Transfer Kernel Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[48]  Motoaki Kawanabe,et al.  Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation , 2007, NIPS.

[49]  Yuan Shi,et al.  Geodesic flow kernel for unsupervised domain adaptation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Zhaohong Deng,et al.  Knowledge-Leverage-Based TSK Fuzzy System Modeling , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  James M. Keller,et al.  Fuzzy Models and Algorithms for Pattern Recognition and Image Processing , 1999 .

[53]  Edwin Lughofer,et al.  SparseFIS: Data-Driven Learning of Fuzzy Systems With Sparsity Constraints , 2010, IEEE Transactions on Fuzzy Systems.

[54]  Ivor W. Tsang,et al.  Domain Transfer Multiple Kernel Learning , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.