Interval Type-2 Fuzzy Restricted Boltzmann Machine

Restricted Boltzmann Machine (RBM) is an energy-based Artificial Neural Network (ANN), applied in several applications like image processing, topic modeling, classification, regression and pattern recognition. The fuzzy version of RBM is a new approach in this field, with parameters considered as fuzzy numbers. In this paper, a fuzzy RBM is extended through interval type-2 membership functions, named the Interval Type-2 Fuzzy RBM (IT2FRBM). The additional uncertainties in the structures of the membership functions are embedded in this model. This is formulated as a maximum likelihood problem which allows the parameters of the type-2 fuzzy numbers to be learned. The capabilities of this proposed approach as a discriminative or generative model are assessed. The robustness of this method against noise is analyzed. The results indicate that this IT2FRBM outperforms RBM and its different fuzzy versions.

[1]  Geoffrey E. Hinton,et al.  Learning a better representation of speech soundwaves using restricted boltzmann machines , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Jongwan Kim,et al.  A fuzzy logic control simulator for adaptive traffic management , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[3]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[4]  Ruslan Salakhutdinov,et al.  Learning Deep Generative Models , 2009 .

[5]  L. A. ZADEH,et al.  The concept of a linguistic variable and its application to approximate reasoning - I , 1975, Inf. Sci..

[6]  Hugo Guterman,et al.  An adaptive neuro-fuzzy system for automatic image segmentation and edge detection , 2002, IEEE Trans. Fuzzy Syst..

[7]  Andri Riid,et al.  Transparent Fuzzy Systems in Modelling and Control , 2003 .

[8]  Yoshiki Uchikawa,et al.  On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm , 1992, IEEE Trans. Neural Networks.

[9]  Rudolf Kruse,et al.  Neuro-fuzzy systems for function approximation , 1999, Fuzzy Sets Syst..

[10]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[11]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Christian Igel,et al.  Training restricted Boltzmann machines: An introduction , 2014, Pattern Recognit..

[14]  Feng Zhao,et al.  Pareto-based interval type-2 fuzzy c-means with multi-scale JND color histogram for image segmentation , 2018, Digit. Signal Process..

[15]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[16]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[17]  C. L. Philip Chen,et al.  Fuzzy Restricted Boltzmann Machine for the Enhancement of Deep Learning , 2015, IEEE Transactions on Fuzzy Systems.

[18]  Geoffrey E. Hinton,et al.  Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine , 2010, NIPS.

[19]  Ehsanollah Kabir,et al.  Introducing a very large dataset of handwritten Farsi digits and a study on their varieties , 2007, Pattern Recognit. Lett..

[20]  Y. Ku,et al.  Introduction to fuzzy arithmetic—theory and applications : Arnold Kaufmann and Madan M. Gupta. 351 pages, diagrams, figures. Van Nostrand Reinhold Company, New York, 1985. , 1986 .

[21]  Alberto Vargas,et al.  Calculating Functions of Interval Type-2 Fuzzy Numbers for Fault Current Analysis , 2007, IEEE Transactions on Fuzzy Systems.

[22]  Xin Zhou,et al.  Image enhancement based on intuitionistic fuzzy sets theory , 2016, IET Image Process..

[23]  Jerry M. Mendel,et al.  Interval Type-2 Fuzzy Logic Systems Made Simple , 2006, IEEE Transactions on Fuzzy Systems.

[24]  Miguel Á. Carreira-Perpiñán,et al.  On Contrastive Divergence Learning , 2005, AISTATS.

[25]  Dongrui Wu,et al.  A type-2 fuzzy logic controller for the liquid-level process , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[26]  Jianbin Qiu,et al.  Recent Advances on Fuzzy-Model-Based Nonlinear Networked Control Systems: A Survey , 2016, IEEE Transactions on Industrial Electronics.

[27]  C. L. Philip Chen,et al.  A Fuzzy Restricted Boltzmann Machine: Novel Learning Algorithms Based on the Crisp Possibilistic Mean Value of Fuzzy Numbers , 2018, IEEE Transactions on Fuzzy Systems.

[28]  Hans Hagen,et al.  Why Interval Arithmetic is so Useful , 2007, Visualization of Large and Unstructured Data Sets.

[29]  Palaiahnakote Shivakumara,et al.  Rough-fuzzy based scene categorization for text detection and recognition in video , 2018, Pattern Recognit..

[30]  Detlef Nauck,et al.  Foundations Of Neuro-Fuzzy Systems , 1997 .

[31]  Hugo Larochelle,et al.  A Neural Autoregressive Topic Model , 2012, NIPS.

[32]  Witold Pedrycz,et al.  Background subtraction using Gaussian-Bernoulli restricted Boltzmann machine , 2018, IET Image Process..

[33]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[34]  Shyamal Kumar Mondal,et al.  A balanced solution of a fuzzy soft set based decision making problem in medical science , 2012, Appl. Soft Comput..

[35]  Tufan Kumbasar,et al.  Learning with Type-2 Fuzzy activation functions to improve the performance of Deep Neural Networks , 2019, Eng. Appl. Artif. Intell..

[36]  Zhi-Qiang Liu,et al.  Type-2 Fuzzy Topic Models for Human Action Recognition , 2015, IEEE Transactions on Fuzzy Systems.

[37]  Mikko Haavisto.  Pretraining Convolutional Neural Networks for Visual Recognition , 2016 .

[38]  Yee Whye Teh,et al.  Rate-coded Restricted Boltzmann Machines for Face Recognition , 2000, NIPS.