Granular transfer learning using type-2 fuzzy HMM for text sequence recognition

Context information plays an important role in text sequence recognition, but it is difficult to harness the uncertainty caused by conflicting implications. In this paper, we propose a novel Granular Transfer (GT) learning with type-2 fuzzy Hidden Markov Model (HMM) called GT2HMM, in which interpretable granules' representation is introduced to describe the contextual uncertainty for its transfer learning. The correspondences among words are transformed into information granules using fuzzy c-means. To fulfill the utilization of granular information in sequence recognition, we construct a type-2 fuzzy HMM which fuses labeled data and unlabeled observations. With a tunable granularity, correspondence information is refined in a coarse-to-fine manner in GT2HMM. Experiments on transductive and inductive transfer learning in part-of-speech (POS) tagging tasks verify the effectiveness of our proposed GT2HMM.

[1]  Witold Pedrycz,et al.  Knowledge transfer in system modeling and its realization through an optimal allocation of information granularity , 2012, Appl. Soft Comput..

[2]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL 2006.

[3]  Witold Pedrycz,et al.  Temporal granulation and its application to signal analysis , 2002, Inf. Sci..

[4]  Foster J. Provost,et al.  Machine learning for targeted display advertising: transfer learning in action , 2013, Machine Learning.

[5]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[6]  Ammar Belatreche,et al.  Adaptive Hidden Markov Model With Anomaly States for Price Manipulation Detection , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[7]  PerlichC.,et al.  Machine learning for targeted display advertising , 2014 .

[8]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[9]  Diane J. Cook,et al.  Transfer Learning across Feature-Rich Heterogeneous Feature Spaces via Feature-Space Remapping (FSR) , 2015, ACM Trans. Intell. Syst. Technol..

[10]  Ivor W. Tsang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 1 Domain Adaptation from Multiple Sources: A Domain- , 2022 .

[11]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[12]  Xuelong Li,et al.  When Location Meets Social Multimedia , 2015, ACM Transactions on Intelligent Systems and Technology.

[13]  Nam Soo Kim,et al.  Maximum a posteriori adaptation of HMM parameters based on speaker space projection , 2004, Speech Commun..

[14]  Peter Christen,et al.  Cross-Language Learning from Bots and Users to Detect Vandalism on Wikipedia , 2015, IEEE Transactions on Knowledge and Data Engineering.

[15]  Philip S. Yu,et al.  Domain Invariant Transfer Kernel Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[16]  Nicolas Ragot,et al.  Combining Structure and Parameter Adaptation of HMMs for Printed Text Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Jeng-Shyang Pan,et al.  Fuzzy Rules Interpolation for Sparse Fuzzy Rule-Based Systems Based on Interval Type-2 Gaussian Fuzzy Sets and Genetic Algorithms , 2013, IEEE Transactions on Fuzzy Systems.

[18]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[19]  Yi Zhang,et al.  Training Conditional Random Fields Using Transfer Learning for Gesture Recognition , 2010, 2010 IEEE International Conference on Data Mining.

[20]  Takanobu Nakahara,et al.  Pattern Mining in POS Data using a Historical Tree , 2006, Sixth IEEE International Conference on Data Mining - Workshops (ICDMW'06).

[21]  Chin-Hui Lee,et al.  Joint maximum a posteriori adaptation of transformation and HMM parameters , 2001, IEEE Trans. Speech Audio Process..

[22]  Jerry M. Mendel,et al.  Type-2 fuzzy sets made simple , 2002, IEEE Trans. Fuzzy Syst..

[23]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[24]  Witold Pedrycz,et al.  Agreement-based fuzzy C-means for clustering data with blocks of features , 2014, Neurocomputing.

[25]  Philip S. Yu,et al.  Adaptation Regularization: A General Framework for Transfer Learning , 2014, IEEE Transactions on Knowledge and Data Engineering.

[26]  Weike Pan,et al.  A survey of transfer learning for collaborative recommendation with auxiliary data , 2016, Neurocomputing.

[27]  I. Burhan Türksen,et al.  Validation criteria for enhanced fuzzy clustering , 2008, Pattern Recognit. Lett..

[28]  Witold Pedrycz,et al.  Granular neural networks , 2001, Neurocomputing.

[29]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[30]  Jia Zeng,et al.  Type-2 fuzzy hidden Markov models and their application to speech recognition , 2006, IEEE Transactions on Fuzzy Systems.

[31]  Andrew McCallum,et al.  Composition of Conditional Random Fields for Transfer Learning , 2005, HLT.

[32]  Amit Konar,et al.  Secondary factor induced stock index time-series prediction using Self-Adaptive Interval Type-2 Fuzzy Sets , 2016, Neurocomputing.

[33]  Sudeep Sarkar,et al.  Improved gait recognition by gait dynamics normalization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Witold Pedrycz,et al.  Time series long-term forecasting model based on information granules and fuzzy clustering , 2015, Eng. Appl. Artif. Intell..

[35]  Witold Pedrycz,et al.  Granular Neural Networks: Concepts and Development Schemes , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[36]  Jerry M. Mendel,et al.  $\alpha$-Plane Representation for Type-2 Fuzzy Sets: Theory and Applications , 2009, IEEE Transactions on Fuzzy Systems.

[37]  C. L. Philip Chen,et al.  Covariance Matrix Estimation with Multi-Regularization Parameters based on MDL Principle , 2012, Neural Processing Letters.

[38]  Qi He,et al.  Tweet Segmentation and Its Application to Named Entity Recognition , 2015, IEEE Transactions on Knowledge and Data Engineering.

[39]  Alexander Clark,et al.  Inducing Syntactic Categories by Context Distribution Clustering , 2000, CoNLL/LLL.

[40]  Nam Soo Kim,et al.  Factored MLLR Adaptation , 2011, IEEE Signal Processing Letters.

[41]  Thorsten Brants,et al.  TnT – A Statistical Part-of-Speech Tagger , 2000, ANLP.

[42]  Oendrila Samanta,et al.  Smoothing of HMM parameters for efficient recognition of online handwriting , 2014, Pattern Recognit..

[43]  Witold Pedrycz,et al.  Granular fuzzy modeling with evolving hyperboxes in multi-dimensional space of numerical data , 2015, Neurocomputing.

[44]  Zhongzhi Shi,et al.  Triplex transfer learning: exploiting both shared and distinct concepts for text classification. , 2014, IEEE transactions on cybernetics.

[45]  J. Mendel,et al.  α-Plane Representation for Type-2 Fuzzy Sets: Theory and Applications , 2009 .