The Reliability of Linear Feature Extractors

This paper introduces the concepts of capacity and reliability of a linear feature extractor. The relationship between the capacity introduced here and the channel capacity in information theory is discussed in some detail. The reliability is associated with the least favorable distributions and gives us a measure of the effectiveness for the worst possible case. It is of particular importance in pattern recognition problems because we have no control over the distributions of the patterns. It is shown that for the family of probability distributions with covariance matrix S, the most reliable (i.e., minimax) feature extractor is the Karhunan-Loève expansion. The concept of reliability is extended to the two-class pattern recognition problem and is discussed in terms of the Bhattacharyya distance.

[1]  Philip M. Lewis,et al.  The characteristic selection problem in recognition systems , 1962, IRE Trans. Inf. Theory.

[2]  T. Kailath.  The Divergence and Bhattacharyya Distance Measures in Signal Selection , 1967 .

[3]  Keinosuke Fukunaga,et al.  Application of the Karhunen-Loève Expansion to Feature Selection and Ordering , 1970, IEEE Trans. Computers.

[4]  George S. Sebestyen,et al.  Recognition of membership in classes , 1961, IRE Trans. Inf. Theory.

[5]  William G. Wee,et al.  On feature selection in a class of distribution-free pattern classifiers , 1970, IEEE Trans. Inf. Theory.

[6]  Thomas W. Calvert,et al.  Nonorthogonal Projections for Feature Extraction in Pattern Recognition , 1969, IEEE Transactions on Computers.

[7]  Demetrios G. Lainiotis,et al.  Comments on linear feature extraction [Corresp.] , 1969, IEEE Trans. Inf. Theory.

[8]  King-Sun Fu,et al.  Selection and Ordering of Feature Observations in a Pattern Recognition System , 1968, Inf. Control..

[9]  TERRY L. HENDERSON,et al.  Application of state-variable techniques to optimal feature extraction - Multichannel analog data , 1970, IEEE Trans. Inf. Theory.

[10]  Matthew Yuschik,et al.  On minimax feature selection and Bhattacharyya coefficients , 1970 .

[11]  Edward A. Patrick,et al.  Nonparametric feature selection , 1969, IEEE Trans. Inf. Theory.

[12]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[13]  D. G. Lainiotis,et al.  COMMENTS ON LINEAR FEATURE EXTRACTION. , 1969 .

[14]  Thomas L. Grettenberg,et al.  Signal selection in communication and radar systems , 1963, IEEE Trans. Inf. Theory.

[15]  Demetrios G. Lainiotis,et al.  A class of upper bounds on probability of error for multihypotheses pattern recognition (Corresp.) , 1969, IEEE Trans. Inf. Theory.

[16]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .