A Simple Rule Extraction Method Using a Compact RBF Neural Network

We propose a simple but efficient method to extract rules from the radial basis function (RBF) neural network. Firstly, the data are classified by an RBF classifier. During training the RBF network, we allow for large overlaps between clusters corresponding to the same class to reduce the number of hidden neurons while maintaining classification accuracy. Secondly, centers of the kernel functions are used as initial conditions when searching for rule premises by gradient descent. Thirdly, redundant rules and unimportant features are removed based on the rule tuning results. Simulations show that our approach results in accurate and concise rules.

[1]  Christopher M. Bishop,et al.  Neural Network for Pattern Recognition , 1995 .

[2]  Michael R. Berthold,et al.  Building precise classifiers with automatic rule extraction , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[3]  Lipo Wang,et al.  Rule extraction from an RBF classifier based on class-dependent features , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[4]  Lipo Wang,et al.  Data Mining With Computational Intelligence , 2006, IEEE Transactions on Neural Networks.

[5]  Lipo Wang,et al.  A GA-based RBF classifier with class-dependent features , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[6]  Christian Pellegrini,et al.  Constraining the MLP power of expression to facilitate symbolic rule extraction , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[7]  Lipo Wang,et al.  Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[8]  John MacIntyre,et al.  Knowledge extraction and insertion from radial basis function networks , 1999 .

[9]  Amit Gupta,et al.  Generalized Analytic Rule Extraction for Feedforward Neural Networks , 1999, IEEE Trans. Knowl. Data Eng..

[10]  M. Glesner,et al.  A new method for generating fuzzy classification systems using RBF neurons with extended RCE learning , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[11]  Lipo Wang,et al.  Training RBF neural networks on unbalanced data , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[12]  Anil Nerode,et al.  Hybrid Knowledge Bases , 1996, IEEE Trans. Knowl. Data Eng..

[13]  T. Kaylani,et al.  A new method for initializing radial basis function classifiers , 1994, Proceedings of IEEE International Conference on Systems, Man and Cybernetics.

[14]  Lipo Wang,et al.  Linguistic Rule Extraction From a Simplified RBF Neural Network , 2001, Comput. Stat..

[15]  Lipo Wang,et al.  Rule extraction by genetic algorithms based on a simplified RBF neural network , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[16]  Xiuju Fu,et al.  Rule Extraction Based on Data Dimensionality Reduction Using RBF Neural Networks , 2001 .

[17]  Asim Roy,et al.  An algorithm to generate radial basis function (RBF)-like nets for classification problems , 1995, Neural Networks.

[18]  Hongjun Lu,et al.  Effective Data Mining Using Neural Networks , 1996, IEEE Trans. Knowl. Data Eng..

[19]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[20]  Rudy Setiono.  Extracting M-of-N rules from trained neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[21]  P. Gubian,et al.  Approximate radial basis function neural networks (RBFNN) to learn smooth relations from noisy data , 1994, Proceedings of 1994 37th Midwest Symposium on Circuits and Systems.