A New Data Mining Scheme Using Artificial Neural Networks

Classification is one of the data mining problems receiving enormous attention in the database community. Although artificial neural networks (ANNs) have been successfully applied in a wide range of machine learning applications, they are however often regarded as black boxes, i.e., their predictions cannot be explained. To enhance the explanation of ANNs, a novel algorithm to extract symbolic rules from ANNs has been proposed in this paper. ANN methods have not been effectively utilized for data mining tasks because how the classifications were made is not explicitly stated as symbolic rules that are suitable for verification or interpretation by human experts. With the proposed approach, concise symbolic rules with high accuracy, that are easily explainable, can be extracted from the trained ANNs. Extracted rules are comparable with other methods in terms of number of rules, average number of conditions for a rule, and the accuracy. The effectiveness of the proposed approach is clearly demonstrated by the experimental results on a set of benchmark data mining classification problems.

[1]  Rudy Setiono.  Extracting M-of-N rules from trained neural networks , 2000, IEEE Trans. Neural Networks Learn. Syst..

[2]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[3]  Rudy Setiono,et al.  Use of a quasi-Newton method in a feedforward neural network construction algorithm , 1995, IEEE Trans. Neural Networks.

[4]  L. Wang,et al.  Application of Data Mining Technology Based on Neural Network in the Engineering , 2007, 2007 International Conference on Wireless Communications, Networking and Mobile Computing.

[5]  Tomasz Imielinski,et al.  Database Mining: A Performance Perspective , 1993, IEEE Trans. Knowl. Data Eng..

[6]  Bart Baesens,et al.  Using Neural Network Rule Extraction and Decision Tables for Credit - Risk Evaluation , 2003, Manag. Sci..

[7]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[8]  Wee Kheng Leow,et al.  FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks , 2004, Applied Intelligence.

[9]  Arjen van Ooyen,et al.  Improving the convergence of the back-propagation algorithm , 1992, Neural Networks.

[10]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[11]  Bernhard Sendhoff,et al.  Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Jadzia Cendrowska,et al.  PRISM: An Algorithm for Inducing Modular Rules , 1987, Int. J. Man Mach. Stud..

[13]  S.K. Anbananthen,et al.  Data Mining using Pruned Artificial Neural Network Tree (ANNT) , 2006, 2006 2nd International Conference on Information & Communication Technologies.

[14]  Jihoon Yang,et al.  Constructive Neural-Network Learning Algorithms for Pattern Classification , 2000 .

[15]  James T. Kwok,et al.  Constructive algorithms for structure learning in feedforward neural networks for regression problems , 1997, IEEE Trans. Neural Networks.

[16]  Rudy Setiono,et al.  A note on knowledge discovery using neural networks and its application to credit card screening , 2009, Eur. J. Oper. Res..

[17]  R. Nakano,et al.  Medical diagnostic expert system based on PDP model , 1988, IEEE 1988 International Conference on Neural Networks.

[18]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[19]  Duc Truong Pham,et al.  RULES: A simple rule extraction system , 1995 .

[20]  Kazuyuki Murase,et al.  A new algorithm to design compact two-hidden-layer artificial neural networks , 2001, Neural Networks.

[21]  Novruz Allahverdi,et al.  Rule extraction from trained adaptive neural networks using artificial immune systems , 2009, Expert Syst. Appl..

[22]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[23]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[24]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[25]  Duc Truong Pham,et al.  An algorithm for automatic rule induction , 1993, Artif. Intell. Eng..

[26]  Wei Shi,et al.  Research on Rules Extraction from Neural Network based on Linear Insertion , 2010, 2010 WASE International Conference on Information Engineering.

[27]  Jude W. Shavlik,et al.  Using neural networks for data mining , 1997, Future Gener. Comput. Syst..

[28]  Guoquan Jiang,et al.  The Research of Data Mining Based on Neural Networks , .

[29]  Rudy Setiono,et al.  Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting , 1997, Neural Computation.

[30]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[31]  Huan Liu,et al.  X2R: a fast rule generator , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[32]  P. Gaur.  Neural Networks in Data Mining , 2012 .

[33]  S. M. Kamruzzaman,et al.  Medical diagnosis using neural network , 2010, ArXiv.

[34]  T. Ash,et al.  Dynamic node creation in backpropagation networks , 1989, International 1989 Joint Conference on Neural Networks.

[35]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[36]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[37]  Jiawei Han,et al.  Efficient and Effective Clustering Methods for Spatial Data Mining , 1994, VLDB.

[38]  Youn-Long Lin,et al.  Performance-driven soft-macro clustering and placement by preserving HDL design hierarchy , 1998, ISPD '98.

[39]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[40]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[41]  Ashish Darbari,et al.  Rule Extraction from Trained ANN: A Survey , 2000 .

[42]  LiMin Fu,et al.  Rule Learning by Searching on Adapted Nets , 1991, AAAI.

[43]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[44]  Huan Liu,et al.  Symbolic Representation of Neural Networks , 1996, Computer.

[45]  Hongjun Lu,et al.  Effective Data Mining Using Neural Networks , 1996, IEEE Trans. Knowl. Data Eng..

[46]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[47]  S. M. Kamruzzaman,et al.  An Algorithm to Extract Rules from Artificial Neural Networks for Medical Diagnosis Problems , 2010, ArXiv.

[48]  Jude W. Shavlik,et al.  Extracting refined rules from knowledge-based neural networks , 2004, Machine Learning.

[49]  Smita. Nirkhi.  Potential use of Artificial Neural Network in Data Mining , 2010, 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE).

[50]  Henrik Jacobsson,et al.  Rule Extraction from Recurrent Neural Networks: ATaxonomy and Review , 2005, Neural Computation.

[51]  Arbee L. P. Chen,et al.  An Efficient Algorithm for Deriving Compact Rules from Databases , 1995, DASFAA.

[52]  Cyril Biryulev,et al.  Research of artificial neural networks usage in data mining and semantic integration , 2010, 2010 Proceedings of VIth International Conference on Perspective Technologies and Methods in MEMS Design.

[53]  Lutz Prechelt,et al.  A Set of Neural Network Benchmark Problems and Benchmarking Rules , 1994 .

[54]  Yoichi Hayashi,et al.  Greedy rule generation from discrete data and its use in neural network rule extraction , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.