Long-term learning for type-2 neural-fuzzy systems

Abstract The development of a new long-term learning framework for interval-valued neural-fuzzy systems is presented for the first time in this article. The need for such a framework is twofold: to address continuous batch learning of data sets, and to take advantage the extra degree of freedom that type-2 Fuzzy Logic systems offer for better model predictive ability. The presented long-term learning framework uses principles of granular computing (GrC) to capture information/knowledge from raw data in the form of interval-valued sets in order to build a computational mechanism that has the ability to adapt to new information in an additive and long-term learning fashion. The latter, is to accommodate new input–output mappings and new classes of data without significantly disturbing existing input–output mappings, therefore maintaining existing performance while creating and integrating new knowledge (rules). This is achieved via an iterative algorithmic process, which involves a two-step operation: iterative rule-base growth (capturing new knowledge) and iterative rule-base pruning (removing redundant knowledge) for type-2 rules. The two-step operation helps create a growing, but sustainable model structure. The performance of the proposed system is demonstrated using a number of well-known non-linear benchmark functions as well as a highly nonlinear multivariate real industrial case study. Simulation results show that the performance of the original model structure is maintained and it is comparable to the updated model's performance following the incremental learning routine. The study is concluded by evaluating the performance of the proposed framework in frequent and consecutive model updates where the balance between model accuracy and complexity is further assessed.

[1]  Chin-Teng Lin,et al.  A TSK-Type-Based Self-Evolving Compensatory Interval Type-2 Fuzzy Neural Network (TSCIT2FNN) and Its Applications , 2014, IEEE Transactions on Industrial Electronics.

[2]  Qian Zhang,et al.  Nature-inspired multi-objective optimisation and transparent knowledge discovery via hierarchical fuzzy modelling , 2008 .

[3]  Jia Zeng,et al.  Type-2 fuzzy Gaussian mixture models , 2008, Pattern Recognit..

[4]  Chin-Teng Lin,et al.  A Mutually Recurrent Interval Type-2 Neural Fuzzy System (MRIT2NFS) With Self-Evolving Structure and Parameters , 2013, IEEE Transactions on Fuzzy Systems.

[5]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[6]  Sameer Singh,et al.  Novelty detection: a review - part 2: : neural network based approaches , 2003, Signal Process..

[7]  V. Vaidehi,et al.  Optimum steepest descent higher level learning radial basis function network , 2015, Expert Syst. Appl..

[8]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[9]  Jerry M. Mendel,et al.  A comparative study of ranking methods, similarity measures and uncertainty measures for interval type-2 fuzzy sets , 2009, Inf. Sci..

[10]  Nikola K. Kasabov,et al.  On-line learning, reasoning, rule extraction and aggregation in locally optimized evolving fuzzy neural networks , 2001, Neurocomputing.

[11]  Oscar Castillo,et al.  Type-1 and Type-2 fuzzy logic controller design using a Hybrid PSO-GA optimization method , 2014, Inf. Sci..

[12]  Paramasivan Saratchandran,et al.  Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction , 2006, Fuzzy Sets Syst..

[13]  Amaury Lendasse,et al.  Autoregressive time series prediction by means of fuzzy inference systems using nonparametric residual variance estimation , 2010, Fuzzy Sets Syst..

[14]  Chia-Feng Juang,et al.  An Interval Type-2 Neural Fuzzy Chip With On-Chip Incremental Learning Ability for Time-Varying Data Sequence Prediction and System Control , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Jerry M. Mendel,et al.  Computing derivatives in interval type-2 fuzzy logic systems , 2004, IEEE Transactions on Fuzzy Systems.

[16]  Rui Pedro Paiva,et al.  Interpretability and learning in neuro-fuzzy systems , 2004, Fuzzy Sets Syst..

[17]  Chia-Feng Juang,et al.  A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning , 2008, IEEE Transactions on Fuzzy Systems.

[18]  Meng Joo Er,et al.  Data driven modeling based on dynamic parsimonious fuzzy neural network , 2013, Neurocomputing.

[19]  Nikola K. Kasabov,et al.  Evolving connectionist systems for adaptive learning and knowledge discovery: Trends and directions , 2015, Knowl. Based Syst..

[20]  Jia Zeng,et al.  Type-2 fuzzy hidden Markov models and their application to speech recognition , 2006, IEEE Transactions on Fuzzy Systems.

[21]  M. Mahfouf,et al.  An Incremental Learning Structure using Granular Computing and Model Fusion With Application to Materials Processing , 2006, 2006 3rd International IEEE Conference Intelligent Systems.

[22]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[23]  Didier Dubois,et al.  Checking the coherence and redundancy of fuzzy knowledge bases , 1997, IEEE Trans. Fuzzy Syst..

[24]  Chia-Feng Juang,et al.  An Interval Type-2 Fuzzy-Neural Network With Support-Vector Regression for Noisy Regression Problems , 2010, IEEE Transactions on Fuzzy Systems.

[25]  Jerry M. Mendel,et al.  Type-2 fuzzy sets made simple , 2002, IEEE Trans. Fuzzy Syst..

[26]  Ali Baraka Soft-computing and human-centric approaches for modelling complex manufacturing systems , 2017 .

[27]  George Panoutsos,et al.  Development of a parsimonious GA-NN ensemble model with a case study for Charpy impact energy prediction , 2011, Adv. Eng. Softw..

[28]  George Panoutsos,et al.  A neural-fuzzy modelling framework based on granular computing: Concepts and applications , 2010, Fuzzy Sets Syst..

[29]  Cheng-Jian Lin,et al.  A self-adaptive neural fuzzy network with group-based symbiotic evolution and its prediction applications , 2006, Fuzzy Sets Syst..

[30]  Yang-Yin Lin,et al.  Dynamic system modeling using a recurrent interval-valued fuzzy neural network and its hardware implementation , 2011, Fuzzy Sets Syst..

[31]  Chin-Teng Lin,et al.  An Interval Type-2 Neural Fuzzy System for Online System Identification and Feature Elimination , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Jerry M. Mendel,et al.  Uncertainty bounds and their use in the design of interval type-2 fuzzy logic systems , 2002, IEEE Trans. Fuzzy Syst..

[33]  L. Zadeh Toward a Perception-Based Theory of Probabilistic Reasoning , 2000, Rough Sets and Current Trends in Computing.

[34]  David Saad,et al.  On-Line Learning in Neural Networks , 1999 .

[35]  Fred Henrik Hamker,et al.  Life-long learning Cell Structures--continuously learning without catastrophic interference , 2001, Neural Networks.

[36]  Kim-Fung Man,et al.  Agent-based evolutionary approach for interpretable rule-based knowledge extraction , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[37]  Hani Hagras,et al.  A type-2 fuzzy embedded agent to realise ambient intelligence in ubiquitous computing environments , 2005, Inf. Sci..

[38]  George Panoutsos,et al.  A Shannon entropy-based conflict measure for enhancing granular computing-based information processing , 2014, 2014 IEEE International Conference on Granular Computing (GrC).

[39]  Vladimir Cherkassky,et al.  Comparison of adaptive methods for function estimation from samples , 1996, IEEE Trans. Neural Networks.

[40]  Xin-She Yang,et al.  A literature survey of benchmark functions for global optimisation problems , 2013, Int. J. Math. Model. Numer. Optimisation.

[41]  Jerry M. Mendel,et al.  Type-2 fuzzy sets and systems: an overview , 2007, IEEE Computational Intelligence Magazine.

[42]  Xingsheng Deng,et al.  Incremental learning of dynamic fuzzy neural networks for accurate system modeling , 2009, Fuzzy Sets Syst..

[43]  Bijan Fazlollahi,et al.  Evolutionary algorithm-based learning of fuzzy neural networks. Part 2: Recurrent fuzzy neural networks , 2009, Fuzzy Sets Syst..

[44]  Byung-In Choi,et al.  Interval type-2 fuzzy membership function generation methods for pattern recognition , 2009, Inf. Sci..

[45]  Jerry M. Mendel,et al.  Interval type-2 fuzzy logic systems , 2000, Ninth IEEE International Conference on Fuzzy Systems. FUZZ- IEEE 2000 (Cat. No.00CH37063).

[46]  Robert Ivor John,et al.  The collapsing method of defuzzification for discretised interval type-2 fuzzy sets , 2009, Inf. Sci..

[47]  Uday S. Dixit,et al.  Application of soft computing techniques in machining performance prediction and optimization: a literature review , 2010 .

[48]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[49]  Guannan Deng,et al.  Monotonic similarity measures between fuzzy sets and their relationship with entropy and inclusion measure , 2016, Fuzzy Sets Syst..

[50]  Chin-Teng Lin,et al.  An online self-constructing neural fuzzy inference network and its applications , 1998, IEEE Trans. Fuzzy Syst..

[51]  George Panoutsos,et al.  Perpetual Learning Framework based on Type-2 Fuzzy Logic System for a Complex Manufacturing Process , 2016 .

[52]  George Panoutsos,et al.  A real-time quality monitoring framework for steel friction stir welding using computational intelligence , 2015 .

[53]  Uzay Kaymak,et al.  Similarity measures in fuzzy rule base simplification , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[54]  Hani Hagras,et al.  Introduction to Interval Type-2 Fuzzy Logic Controllers - Towards Better Uncertainty Handling in Real World Applications , 2009 .

[55]  I-Cheng Yeh,et al.  Modeling of strength of high-performance concrete using artificial neural networks , 1998 .

[56]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[57]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[58]  Chi-Hsu Wang,et al.  Dynamical optimal training for interval type-2 fuzzy neural network (T2FNN) , 2003, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[59]  Xiang Li,et al.  Online probabilistic learning for fuzzy inference system , 2014, Expert Syst. Appl..

[60]  George Panoutsos,et al.  Interval Type-2 Radial Basis Function Neural Network: A Modeling Framework , 2015, IEEE Transactions on Fuzzy Systems.

[61]  Zhihua Cui,et al.  Swarm Intelligence and Bio-Inspired Computation: Theory and Applications , 2013 .