Mismatch in the classification of linear subspaces: Upper bound to the probability of error

This paper studies the performance associated with the classification of linear subspaces corrupted by noise with a mismatched classifier. In particular, we consider a problem where the classifier observes a noisy signal, the signal distribution conditioned on the signal class is zero-mean Gaussian with low-rank covariance matrix, and the classifier knows only the mismatched parameters in lieu of the true parameters. We derive an upper bound to the misclassification probability of the mismatched classifier and characterize its behaviour. Specifically, our characterization leads to sharp sufficient conditions that describe the absence of an error floor in the low-noise regime, and that can be expressed in terms of the principal angles and the overlap between the true and the mismatched signal subspaces.

[1]  Hermann Ney,et al.  Novel tight classification error bounds under mismatch conditions based on f-Divergence , 2013, 2013 IEEE Information Theory Workshop (ITW).

[2]  Neil D. Lawrence,et al.  Dataset Shift in Machine Learning , 2009 .

[3]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[4]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ata Kabán,et al.  Label-Noise Robust Logistic Regression and Its Applications , 2012, ECML/PKDD.

[7]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[8]  Demetrios Kazakos Signal detection under mismatch , 1982, IEEE Trans. Inf. Theory.

[9]  Golub Gene H. Et.Al Matrix Computations, 3rd Edition , 2007 .

[10]  Sergio Verdú Mismatched estimation and relative entropy , 2010, IEEE Trans. Inf. Theory.

[11]  Tamás Linder,et al.  Mismatch in high-rate entropy-constrained vector quantization , 2003, IEEE Trans. Inf. Theory.

[12]  Anelia Somekh-Baruch A General Formula for the Mismatch Capacity , 2015, IEEE Transactions on Information Theory.

[13]  Emre Telatar,et al.  Mismatched decoding revisited: General alphabets, channels with memory, and the wide-band limit , 2000, IEEE Trans. Inf. Theory.

[14]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[15]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[16]  Nagarajan Natarajan,et al.  Learning with Noisy Labels , 2013, NIPS.

[17]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.