Extracting rules for classification problems: AIS based approach

Although Artificial Neural Network (ANN) usually reaches high classification accuracy, the obtained results in most cases may be incomprehensible. This fact is causing a serious problem in data mining applications. The rules that are derived from ANN are needed to be formed to solve this problem and various methods have been improved to extract these rules. In our previous work, a hybrid neural network was presented for classification (Kahramanli & Allahverdi, 2008). In this study a method that uses Artificial Immune Systems (AIS) algorithm has been presented to extract rules from trained hybrid neural network. The data were obtained from the University of California at Irvine (UCI) machine learning repository. The datasets are Cleveland heart disease and Hepatitis data. The proposed method achieved accuracy values 96.4% and 96.8% for Cleveland heart disease dataset and Hepatitis dataset respectively. It is been observed that these results are one of the best results comparing with results obtained from related previous studies and reported in UCI web sites.

[1]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[2]  Michael C. Mozer,et al.  Rule Induction through Integrated Symbolic and Subsymbolic Processing , 1991, NIPS.

[3]  Manoj Kumar Tiwari,et al.  Psycho-Clonal algorithm based approach to solve continuous flow shop scheduling problem , 2006, Expert Syst. Appl..

[4]  Edward Keedwell,et al.  CREATING RULES FROM TRAINED NEURAL NETWORKS USING GENETIC ALGORITHMS , 2000 .

[5]  José Miguel Mantas,et al.  Extraction of similarity based fuzzy rules from artificial neural networks , 2006, Int. J. Approx. Reason..

[6]  E. Keedwell,et al.  Evolving rules from neural networks trained on continuous data , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[7]  Hao Xing,et al.  Extract intelligible and concise fuzzy rules from neural networks , 2002, Fuzzy Sets Syst..

[8]  Yoichi Hayashi,et al.  Greedy rule generation from discrete data and its use in neural network rule extraction , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[9]  Zhi-Hua Zhou,et al.  Rule learning based on neural network ensemble , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[10]  H. Narazaki,et al.  A method for extracting approximate rules from neural network , 1995, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[11]  Jacek M. Zurada,et al.  Extraction of rules from artificial neural networks for nonlinear regression , 2002, IEEE Trans. Neural Networks.

[12]  Wlodzislaw Duch,et al.  A new methodology of extraction, optimization and application of crisp and fuzzy logical rules , 2001, IEEE Trans. Neural Networks.

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Snehasis Mukhopadhyay,et al.  A comparative study of genetic sequence classification algorithms , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[15]  A. Pazos,et al.  Automatic recurrent ANN rule extraction with genetic programming , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[16]  Peter Géczy,et al.  Rule Extraction from Trained Artificial Neural Networks , 1997, ICONIP.

[17]  Jonathan Timmis,et al.  Artificial Immune Systems: A New Computational Intelligence Approach , 2003 .

[18]  H. Jaap van den Herik,et al.  Interpretable Neural Networks with BP-SOM , 1998, ECML.

[19]  J.M.T. Romano,et al.  MLP-Based Equalization and Pre-Distortion Using an Artificial Immune Network , 2005, 2005 IEEE Workshop on Machine Learning for Signal Processing.

[20]  Jude W. Shavlik,et al.  Extracting Refined Rules from Knowledge-Based Neural Networks , 1993, Machine Learning.

[21]  Nurhan Karaboga,et al.  Artificial immune algorithm for IIR filter design , 2005, Eng. Appl. Artif. Intell..

[22]  Jonathan Timmis,et al.  Artificial immune systems - a new computational intelligence paradigm , 2002 .

[23]  M. Esmel ElAlami,et al.  Extracting rules from trained neural network using GA for managing E-business , 2004, Appl. Soft Comput..

[24]  Krysia Broda,et al.  Symbolic knowledge extraction from trained neural networks: A sound approach , 2001, Artif. Intell..

[25]  J.A. Ramirez,et al.  A modified immune network algorithm for multimodal electromagnetic problems , 2006, IEEE Transactions on Magnetics.

[26]  Chu Kiong Loo,et al.  Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP , 2005, IEEE Transactions on Knowledge and Data Engineering.

[27]  Novruz Allahverdi,et al.  Design of a hybrid system for the diabetes and heart diseases , 2008, Expert Syst. Appl..

[28]  Jonathan Timmis,et al.  A Comment on Opt-AiNET: An Immune Network Algorithm for Optimisation , 2004, GECCO.

[29]  Shouhong Wang,et al.  Classification with incomplete survey data: a Hopfield neural network approach , 2005, Comput. Oper. Res..

[30]  Wee Kheng Leow,et al.  FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks , 2004, Applied Intelligence.

[31]  Stephen I. Gallant,et al.  Connectionist expert systems , 1988, CACM.

[32]  Donald C. Wunsch,et al.  Neural network explanation using inversion , 2007, Neural Networks.

[33]  Petr Musílek,et al.  Immune programming , 2006, Inf. Sci..

[34]  Hamid Mohamadi,et al.  Data mining with a simulated annealing based fuzzy classification system , 2008, Pattern Recognit..

[35]  Saeid Nahavandi,et al.  Learning to detect texture objects by artificial immune approaches , 2004, Future Gener. Comput. Syst..

[36]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks. I. Classification , 1992, IEEE Trans. Neural Networks.

[37]  Pascal Bouvry,et al.  Anomaly detection in TCP/IP networks using immune systems paradigm , 2007, Comput. Commun..

[38]  Vasile Palade,et al.  Rule extraction from neural networks by interval propagation , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[39]  Hongjun Lu,et al.  Effective Data Mining Using Neural Networks , 1996, IEEE Trans. Knowl. Data Eng..

[40]  P. K. Simpson.  Fuzzy Min-Max Neural Networks-Part 1 : Classification , 1992 .

[41]  Tung-Hsu Hou,et al.  Using neural networks and immune algorithms to find the optimal parameters for an IC wire bonding process , 2008, Expert Syst. Appl..

[42]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[43]  Michael C. Mozer,et al.  Dynamic On-line Clustering and State Extraction: An Approach to Symbolic Learning , 1998, Neural Networks.

[44]  P. K. Simpson,et al.  Fuzzy min-max neural networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[45]  Nelson F. F. Ebecken,et al.  Extracting rules from multilayer perceptrons in classification problems: A clustering-based approach , 2006, Neurocomputing.

[46]  Michael C. Mozer,et al.  Template-Based Algorithms for Connectionist Rule Extraction , 1994, NIPS.