Class-Specific Kernel Discriminant Analysis Revisited: Further Analysis and Extensions

In this paper, we revisit class-specific kernel discriminant analysis (KDA) formulation, which has been applied in various problems, such as human face verification and human action recognition. We show that the original optimization problem solved for the determination of class-specific discriminant projections is equivalent to a low-rank kernel regression (LRKR) problem using training data-independent target vectors. In addition, we show that the regularized version of class-specific KDA is equivalent to a regularized LRKR problem, exploiting the same targets. This analysis allows us to devise a novel fast solution. Furthermore, we devise novel incremental, approximate and deep (hierarchical) variants. The proposed methods are tested in human facial image and action video verification problems, where their effectiveness and efficiency is shown.

[1]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[2]  Alexandros Iosifidis,et al.  On the Optimal Class Representation in Linear Discriminant Analysis , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Alexandros Iosifidis,et al.  Activity-Based Person Identification Using Fuzzy Representation and Discriminant Learning , 2012, IEEE Transactions on Information Forensics and Security.

[4]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[5]  G. Stewart.  Matrix Algorithms, Volume II: Eigensystems , 2001 .

[6]  Shaoning Pang,et al.  Incremental linear discriminant analysis for classification of data streams , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[7]  Rong Jin,et al.  Scalable Kernel Clustering: Approximate Kernel k-means , 2014, ArXiv.

[8]  Feiping Nie,et al.  Trace Ratio Problem Revisited , 2009, IEEE Transactions on Neural Networks.

[9]  Wai Keung Wong,et al.  Deep Learning Regularized Fisher Mappings , 2011, IEEE Transactions on Neural Networks.

[10]  Anastasios Tefas,et al.  Class-Specific Kernel-Discriminant Analysis for Face Verification , 2007, IEEE Transactions on Information Forensics and Security.

[11]  Xuesong Lu,et al.  Fisher Discriminant Analysis With L1-Norm , 2014, IEEE Transactions on Cybernetics.

[12]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[13]  Alexandros Iosifidis,et al.  Class-Specific Reference Discriminant Analysis With Application in Human Behavior Analysis , 2015, IEEE Transactions on Human-Machine Systems.

[14]  Alexandros Iosifidis,et al.  Scaling Up Class-Specific Kernel Discriminant Analysis for Large-Scale Face Verification , 2016, IEEE Transactions on Information Forensics and Security.

[15]  J KriegmanDavid,et al.  Acquiring Linear Subspaces for Face Recognition under Variable Lighting , 2005 .

[16]  Josef Kittler,et al.  Class-Specific Kernel Fusion of Multiple Descriptors for Face Verification Using Multiscale Binarised Statistical Image Features , 2014, IEEE Transactions on Information Forensics and Security.

[17]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[18]  Juan Carlos Niebles,et al.  Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification , 2010, ECCV.

[19]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[20]  Michael K. Ng,et al.  Incremental Linear Discriminant Analysis: A Fast Algorithm and Comparisons , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[21]  Takeo Kanade,et al.  Comprehensive database for facial expression analysis , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[22]  Jiwen Lu,et al.  Automatic Subspace Learning via Principal Coefficients Embedding , 2014, IEEE Transactions on Cybernetics.

[23]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[24]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[25]  KittlerJosef,et al.  Class-Specific Kernel Fusion of Multiple Descriptors for Face Verification Using Multiscale Binarised Statistical Image Features , 2014 .

[26]  Pong C. Yuen,et al.  Incremental Linear Discriminant Analysis for Face Recognition , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Anastasios Maronidis,et al.  Subclass Graph Embedding and a Marginal Fisher Analysis paradigm , 2015, Pattern Recognit..

[28]  Rasmus Pagh,et al.  Fast and scalable polynomial kernels via explicit feature maps , 2013, KDD.

[29]  C. Ding,et al.  On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions , 2013, KDD.

[30]  Ke Lu,et al.  Low-Rank Discriminant Embedding for Multiview Learning , 2017, IEEE Transactions on Cybernetics.

[31]  Cordelia Schmid,et al.  Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[32]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[33]  Alexandros Iosifidis,et al.  Multi-view human movement recognition based on fuzzy distances and linear discriminant analysis , 2012, Comput. Vis. Image Underst..

[34]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[35]  Xuelong Li,et al.  Graph Regularized Non-Negative Low-Rank Matrix Factorization for Image Clustering , 2017, IEEE Transactions on Cybernetics.

[36]  Stan Z. Li,et al.  Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition , 2016, IEEE Transactions on Cybernetics.

[37]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[39]  Alexandros Iosifidis,et al.  Kernel Reference Discriminant Analysis , 2014, Pattern Recognit. Lett..

[40]  Michael J. Lyons,et al.  Coding facial expressions with Gabor wavelets , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[41]  André Stuhlsatz,et al.  Feature Extraction With Deep Neural Networks by a Generalized Discriminant Analysis , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Jieping Ye,et al.  Least squares linear discriminant analysis , 2007, ICML '07.

[43]  Aleix M. Martínez,et al.  Subclass discriminant analysis , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Stefanos Zafeiriou,et al.  Regularized Kernel Discriminant Analysis With a Robust Kernel for Face Recognition and Verification , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[46]  G. F. Hughes,et al.  On the mean accuracy of statistical pattern recognizers , 1968, IEEE Trans. Inf. Theory.

[47]  Fadi Dornaika,et al.  Exponential Local Discriminant Embedding and Its Application to Face Recognition , 2013, IEEE Transactions on Cybernetics.

[48]  Alexandros Iosifidis,et al.  Class-Specific Nonlinear Projections Using Class-Specific Kernel Spaces , 2015, 2015 IEEE Trustcom/BigDataSE/ISPA.

[49]  Lawrence K. Saul,et al.  Kernel Methods for Deep Learning , 2009, NIPS.

[50]  G. W. Stewart,et al.  Matrix Algorithms: Volume 1, Basic Decompositions , 1998 .

[51]  Cordelia Schmid,et al.  Actions in context , 2009, CVPR.

[52]  Witold Pedrycz,et al.  Incremental Hashing for Semantic Image Retrieval in Nonstationary Environments , 2017, IEEE Transactions on Cybernetics.

[53]  Cordelia Schmid,et al.  Action Recognition with Improved Trajectories , 2013, 2013 IEEE International Conference on Computer Vision.

[54]  Nojun Kwak.  Implementing Kernel Methods Incrementally by Incremental Nonlinear Projection Trick , 2017, IEEE Transactions on Cybernetics.

[55]  Bin Xu,et al.  Generalized Discriminant Analysis: A Matrix Exponential Approach , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[56]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[57]  Alexandros Iosifidis,et al.  Graph Embedded Extreme Learning Machine , 2016, IEEE Transactions on Cybernetics.

[58]  Alexandros Iosifidis,et al.  Generalized Multi-View Embedding for Visual Recognition and Cross-Modal Retrieval , 2016, IEEE Transactions on Cybernetics.

[59]  Jiawei Han,et al.  Speed up kernel discriminant analysis , 2011, The VLDB Journal.