A technique for feature selection in multiclass problems

One of the main phases in the development of a system for the classification of remote sensing images is the definition of an effective set of features to be given as input to the classifier. In particular, it is often useful to reduce the number of features available, while saving the possibility to discriminate among the different land-cover classes to be recognized. This paper addresses this topic with reference to applications that involve more than two land-cover classes (multiclass problems). Several criteria proposed in the remote sensing literature are considered and compared with one another and with the criterion presented by the authors. Such a criterion, unlike those usually adopted for multiclass problems, is related to an upper bound to the error probability of the Bayes classifier. As the objective of feature selection is generally to identify a reduced set of features that minimize the errors of the classifier, the aforementioned property is very important because it allows one to select features by taking into account their effects on classification errors. Experiments on two remote sensing datasets are described and discussed. These experiments confirm the effectiveness of the proposed criterion, which performs slightly better than all the others considered in the paper. In addition, the results obtained provide useful information about the behaviour of different classical criteria when applied in multiclass cases.

[1]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[2]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognition Letters.

[3]  M. E. Jernigan,et al.  Texture Analysis and Discrimination in Additive Noise , 1990, Comput. Vis. Graph. Image Process..

[4]  Philip H. Swain,et al.  Remote Sensing: The Quantitative Approach , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  T. Kailath The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[6]  F.D. Garber,et al.  Bounds on the Bayes Classification Error Based on Pairwise Risk Functions , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Josef Kittler,et al.  Divergence Based Feature Selection for Multimodal Class Densities , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Robert A. Schowengerdt,et al.  Remote sensing, models, and methods for image processing , 1997 .

[9]  John A. Richards,et al.  Remote Sensing Digital Image Analysis , 1986 .

[10]  Lorenzo Bruzzone,et al.  An extension of the Jeffreys-Matusita distance to multiclass cases for feature selection , 1995, IEEE Trans. Geosci. Remote. Sens..

[11]  D. Lainiotis,et al.  Probability of Error Bounds , 1971 .

[12]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  S. Krishnan,et al.  Feature selection for pattern classification with Gaussian mixture models: A new objective criterion , 1996, Pattern Recognit. Lett..

[14]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[15]  Martin E. Hellman,et al.  Probability of error, equivocation, and the Chernoff bound , 1970, IEEE Trans. Inf. Theory.

[16]  I. L. Thomas,et al.  Review Article A review of multi-channel indices of class separability , 1987 .

[17]  D. Lainiotis A class of upper-bounds on probability of error for multi-hypotheses pattern recognition , 1969 .

[18]  Demetrios G. Lainiotis,et al.  A class of upper bounds on probability of error for multihypotheses pattern recognition (Corresp.) , 1969, IEEE Trans. Inf. Theory.

[19]  David A. Landgrebe,et al.  Feature Extraction Based on Decision Boundaries , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[21]  Pierre A. Devijver,et al.  On a New Class of Bounds on Bayes Risk in Multihypothesis Pattern Recognition , 1974, IEEE Transactions on Computers.