Integration of Statistical and Neural Methods to Design Classifiers in Case of Unequal Covariance Matrices

The paper extends integration of statistical and neural approaches to design linear classifiers for situations where covariance matrices differ essentially. To ensure a good start for training of the single layer perceptron we perform special data shift and rotation. Here a difference between the covariance matrices of distinct pattern classes plays essential role. After the data transformation, we obtain a good linear classifier just after the first batch-mode training iteration. In small training set size cases, one needs utilize simplified estimates of covariance matrices.

[1]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[2]  Akihiko Miyake,et al.  Mathematical aspects of optimal linear discriminant function , 1979, COMPSAC.

[3]  D. W. Peterson,et al.  A method of finding linear discriminant functions for a class of performance criteria , 1966, IEEE Trans. Inf. Theory.

[4]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[5]  R. E. Warmack,et al.  An Algorithm for the Optimal Solution of Linear Inequalities and its Application to Pattern Recognition , 1973, IEEE Transactions on Computers.

[6]  A. C. Wolff.  The estimation of the optimum linear decision function with a sequential random method , 1966, IEEE Trans. Inf. Theory.

[7]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[8]  Hai Do-Tu,et al.  Learning Algorithms for Nonparametric Solution to the Minimum Error Classification Problem , 1978, IEEE Transactions on Computers.

[9]  R. Casey,et al.  Advances in Pattern Recognition , 1971 .

[10]  Toshihide Ibaraki,et al.  Adaptive Linear Classifier by Linear Programming , 1970, IEEE Trans. Syst. Sci. Cybern..

[11]  Robert P. W. Duin,et al.  K-nearest Neighbors Directed Noise Injection in Multilayer Perceptron Training , 2000, IEEE Trans. Neural Networks Learn. Syst..

[12]  Sarunas Raudys,et al.  Structures of the Covariance Matrices in the Classifier Design , 1998, SSPR/SPR.

[13]  Ausra Saudargiene.  Structurization of the Covariance Matrix by Process Type and Block-Diagonal Models in the Classifier Design , 1999, Informatica.

[14]  Sarunas Raudys,et al.  Evolution and generalization of a single neurone: : II. Complexity of statistical classifiers and sample size considerations , 1998, Neural Networks.

[15]  Fred W. Smith.  Design of minimum-error optimal classifiers for patterns from distributions with Gaussian tails , 1971, IEEE Trans. Inf. Theory.

[16]  T. W. Anderson,et al.  Classification into two Multivariate Normal Distributions with Different Covariance Matrices , 1962 .

[17]  Sarunas Raudys,et al.  First-Order Tree-Type Dependence between Variables and Classification Performance , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  S. Raudys,et al.  Effect of initial values in simple perception , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[19]  Sarunas Raudys,et al.  Evolution and generalization of a single neurone: I. Single-layer perceptron as seven statistical classifiers , 1998, Neural Networks.

[20]  N. E. Day.  Linear and quadratic discrimination in pattern recognition (Corresp.) , 1969, IEEE Trans. Inf. Theory.

[21]  Sarunas Raudys.  How good are support vector machines? , 2000, Neural Networks.

[22]  Stephen S. Yau,et al.  Design of Pattern Classifiers with the Updating Property Using Stochastic Approximation Techniques , 1968, IEEE Transactions on Computers.

[23]  David G. Stork,et al.  Pattern Classification , 1973 .