Dimensionality optimization by heuristic greedy learning vs. genetic algorithms in knowledge discovery and data mining

Dimensionality optimization involves optimizing the size of data sets from both dimensions, variable and observation selections. The ultimate objective of dimensionality optimization is to obtain the induced data space, by reducing both dimensionalities in such a way that the reduced subset could retain sufficient information. In most real-world applications, it is not known what the best subset is and what should be contained in such a subset. Selecting the appropriate subset is extremely important in effectively mining over large data sets in the sense that it is the only source for any data mining and knowledge discovery algorithm to work with the data of interest reliably.The statistical as well as artificial intelligence community has provided good methods in this domain, but still a lot of improvements need to be made, especially for data mining applications. This paper introduces a heuristic methodology that integrates heuristic greedy search methods and tree-structured SampleC4.5 to efficiently find the optimal subset of very large data sets from both dimensions simultaneously. A GA-based optimization approach is also proposed in the paper. Experimental results are presented which illustrate the effectiveness of our approaches in digging out the important underlying patterns, and indicate the potential advantages of the proposed techniques to improve the optimizing process while staying out of misleading dilemma. The results of our experiments also show the robustness of our approaches and complementary characteristics for knowledge discovery and data mining tasks.

[1]  Kenneth de Jong,et al.  Adaptive System Design: A Genetic Approach , 1980, IEEE Trans. Syst. Man Cybern..

[2]  Qiwen Wang,et al.  A neural network model for the wire bonding process , 1993, Comput. Oper. Res..

[3]  Albert Donally Bethke,et al.  Genetic Algorithms as Function Optimizers , 1980 .

[4]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[5]  K. De Jong Learning with Genetic Algorithms: An Overview , 1988 .

[6]  Jerzy W. Bala,et al.  Hybrid Learning Using Genetic Algorithms and Decision Trees for Pattern Classification , 1995, IJCAI.

[7]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[8]  William Frawley,et al.  Knowledge Discovery in Databases , 1991 .

[9]  George A. Marcoulides,et al.  Multivariate Statistical Methods: A First Course , 1997 .

[10]  K. De Jong Adaptive System Design: A Genetic Approach , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[11]  Kenneth A. De Jong,et al.  Genetic algorithms as a tool for feature selection in machine learning , 1992, Proceedings Fourth International Conference on Tools with Artificial Intelligence TAI '92.

[12]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[13]  Moustafa Ghanem,et al.  Large Scale Data Mining: Challenges and Responses , 1997, KDD.

[14]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[15]  P. J. W. Noble Programming the machine , 1989 .

[16]  Bruce L. Golden,et al.  Predicting the success of nations at the Summer Olympics using neural networks , 1999, Comput. Oper. Res..