Neural network based class-conditional probability density function using kernel trick for supervised classifier

The practical limitation of the Bayes classifier used in pattern recognition is computing the class-conditional probability density function (pdf) of the vectors belonging to the corresponding classes. In this paper, a neural network based approach is proposed to model the class-conditional pdf, which can be further used in supervised classifier (e.g. Bayes classifier). It is also suggested to use kernel version (using kernel trick) of the proposed approach to obtain the class-conditional pdf of the corresponding training vectors in the higher dimensional space. This is used for better class separation and hence better classification rate is achieved. The performance of the proposed technique is validated by using the synthetic data and the real data. The simulation results show that the proposed technique on synthetic data and the real data performs well (in terms of classification accuracy) when compared with the classical Fisher?s Linear Discriminant Analysis (LDA) and Gaussian based Kernel-LDA.

[1]  P. Palanisamy,et al.  Formulating Particle Swarm Optimization based Generalized Kernel Function for Kernel-Linear Discriminant Analysis , 2012 .

[2]  Amir Sarajedini,et al.  Conditional probability density function estimation with sigmoidal neural networks , 1999, IEEE Trans. Neural Networks.

[3]  Badong Chen,et al.  Mean square convergence analysis for kernel least mean square algorithm , 2012, Signal Process..

[4]  Weifeng Liu,et al.  Kernel least mean square algorithm with constrained growth , 2009, Signal Process..

[5]  Haixia Xu,et al.  Adaptive kernel principal component analysis , 2010, Signal Process..

[6]  Zhenmin Tang,et al.  A modification of kernel discriminant analysis for high-dimensional data - with application to face recognition , 2010, Signal Process..

[7]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[8]  Haesun Park,et al.  A comparison of generalized linear discriminant analysis algorithms , 2008, Pattern Recognit..

[9]  J. Cid-Sueiro,et al.  Neural architectures for parametric estimation of a posteriori probabilities by constrained conditional density functions , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[10]  Xianda Zhang,et al.  Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems , 2010, Signal Process..

[11]  Hermann Ney,et al.  On the Probabilistic Interpretation of Neural Network Classifiers and Discriminative Training Criteria , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Ziqiang Wang,et al.  Multiple kernel local Fisher discriminant analysis for face recognition , 2013, Signal Process..

[13]  Yuanqing Li,et al.  A linear discriminant analysis method based on mutual information maximization , 2011, Pattern Recognit..

[14]  Aristidis Likas,et al.  Shared kernel models for class conditional density estimation , 2001, IEEE Trans. Neural Networks.

[15]  Masahiro Tanaka,et al.  Pattern classification by stochastic neural network with missing data , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).