Bayesian network classifiers versus selective k-NN classifier

In this paper Bayesian network classifiers are compared to the k-nearest neighbor (k-NN) classifier, which is based on a subset of features. This subset is established by means of sequential feature selection methods. Experimental results on classifying data of a surface inspection task and data sets from the UCI repository show that Bayesian network classifiers are competitive with selective k-NN classifiers concerning classification accuracy. The k-NN classifier performs well in the case where the number of samples for learning the parameters of the Bayesian network is small. Bayesian network classifiers outperform selective k-NN methods in terms of memory requirements and computational demands. This paper demonstrates the strength of Bayesian networks for classification.

[1]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[2]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[3]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[4]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[5]  Franz Pernkopf,et al.  Feature Selection for Classification Using Genetic Algorithms with a Novel Encoding , 2001, CAIP.

[6]  Jack Perkins,et al.  Pattern recognition in practice , 1980 .

[7]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[8]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[9]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[10]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[11]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[12]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[13]  Dr. Zbigniew Michalewicz,et al.  How to Solve It: Modern Heuristics , 2004 .

[14]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[15]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[16]  J. Kittler,et al.  Feature Set Search Alborithms , 1978 .

[17]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[18]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[19]  Eamonn J. Keogh,et al.  Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches , 1999, AISTATS.

[20]  Michael I. Jordan,et al.  Probabilistic Networks and Expert Systems , 1999 .

[21]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[22]  Thomas Roß,et al.  Feature selection for optimized skin tumor recognition using genetic algorithms , 1999, Artif. Intell. Medicine.

[23]  Franz Pernkopf,et al.  Visual Inspection of Machined Metallic High-Precision Surfaces , 2002, EURASIP J. Adv. Signal Process..

[24]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Ron Kohavi,et al.  Irrelevant Features and the Subset Selection Problem , 1994, ICML.

[26]  Chi Hau Chen,et al.  Pattern recognition and signal processing , 1978 .

[27]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[28]  Ron Kohavi,et al.  Feature Subset Selection Using the Wrapper Method: Overfitting and Dynamic Search Space Topology , 1995, KDD.

[29]  David G. Stork,et al.  Pattern Classification , 1973 .

[30]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[31]  Francesc J. Ferri,et al.  Comparative study of techniques for large-scale feature selection* *This work was suported by a SERC grant GR/E 97549. The first author was also supported by a FPI grant from the Spanish MEC, PF92 73546684 , 1994 .

[32]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[33]  Gregory M. Provan,et al.  Efficient Learning of Selective Bayesian Network Classifiers , 1996, ICML.

[34]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[35]  Franz Pernkopf,et al.  Floating search algorithm for structure learning of Bayesian network classifiers , 2003, Pattern Recognit. Lett..

[36]  Franz Pernkopf,et al.  Detection of surface defects on raw steel blocks using Bayesian network classifiers , 2004, Pattern Analysis and Applications.

[37]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognit. Lett..

[38]  Pavel Paclík,et al.  Adaptive floating search methods in feature selection , 1999, Pattern Recognit. Lett..

[39]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[40]  Anil K. Jain,et al.  Algorithms for feature selection: An evaluation , 1996, Proceedings of 13th International Conference on Pattern Recognition.