Learning fuzzy rules through ant optimization, LASSO and Dirichlet Mixtures

In the area of fuzzy systems, one of the main problems is finding the set of rules that can give us the best results in specific problems. Further, the finding of this set is a combinatorial problem. There are several techniques for building these sets, but it is possible to group them in two main classes: The bottom-up approaches and the top-down approaches. This work proposes a new top-down approach to the fuzzy systems learning based in clustering and optimization techniques. The algorithm is split in two stages: First, it determines the fuzzy sets of each input and output linguistic variable, and second, it calculates the fuzzy rules from the obtained fuzzy sets. For the first part, a Dirichlet Mixture (DM) is used to cluster data to assign a fuzzy sets to each new cluster, since a fuzzy set can be seen as a generalized probability function, and hence the fuzzy sets of a given linguistic variable can be seen as a mixture of probabilities (a Gaussian Mixture). Then, an optimization problem is solved by using Ant Colony Optimization (ACO) to generate the minimum set of possible rules for classification by using a version of the Least Absolute Shrinkage and Selection Operator(LASSO) for the fitness function. This ACO was implemented in a CUDA GPU to deal with the combinatorial problem of rule generation. Finally, this new algorithm is used to attack the problem of color image segmentation.

[1]  E. H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Man Mach. Stud..

[2]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[3]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  G. Alonso.  Segmentación de Imágenes con Algoritmos de Agrupamiento Utilizando la Base de Datos BSDS500 "The Berkeley Segmentation Dataset and Benchmark , 2016 .

[5]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[6]  Philip S. Yu,et al.  Evolutionary Clustering by Hierarchical Dirichlet Process with Hidden Markov State , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[7]  R. K. Gupta,et al.  Study on Ductility of Ti Aluminides Using Mamdani Fuzzy Inference System , 2011, SocProS.

[8]  Jie Cheng,et al.  Programming Massively Parallel Processors. A Hands-on Approach , 2010, Scalable Comput. Pract. Exp..

[9]  Radford M. Neal.  Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[10]  Swarup Medasani,et al.  Categorization of image databases for efficient retrieval using robust mixture decomposition , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[11]  Lawrence Carin,et al.  Dirichlet process mixture models with multiple modalities , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[13]  Mark Zwolinski,et al.  Mutual Information Theory for Adaptive Mixture Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Marco Dorigo,et al.  Distributed Optimization by Ant Colonies , 1992 .

[15]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[16]  John W. Pratt,et al.  A Normal Approximation for Binomial, F, Beta, and other Common, Related Tail Probabilities, II , 1968 .

[17]  Philip S. Yu,et al.  Dirichlet Process Based Evolutionary Clustering , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[18]  Paul D. Gader,et al.  Endmember detection using the Dirichlet process , 2008, 2008 19th International Conference on Pattern Recognition.

[19]  D. Aldous.  Exchangeability and related topics , 1985 .

[20]  Nizar Bouguila,et al.  Unsupervised learning of a finite mixture model based on the Dirichlet distribution and its application , 2004, IEEE Transactions on Image Processing.

[21]  Samuel J. Gershman,et al.  A Tutorial on Bayesian Nonparametric Models , 2011, 1106.2697.

[22]  Chia-Feng Juang,et al.  A self-generating fuzzy system with ant and particle swarm cooperative optimization , 2009, Expert Syst. Appl..

[23]  Mohamed Chtourou,et al.  A new method for fuzzy rule base reduction , 2013, J. Intell. Fuzzy Syst..

[24]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[25]  Jorge Casillas,et al.  Learning Fuzzy Rules Using Ant Colony Optimization Algorithms , 2000 .