Fisher's Discriminant with Natural Image Priors

Linear discriminant analysis that takes spatial smoothness into account has been developed and widely used in image processing society. However, two questions remain unanswered. First, which is the best way to incorporate the smoothness property of images with linear discriminant analysis? Second, which is the best representation for the smoothness property of images? To answer the first question, we propose a Bayesian framework of Gaussian process in order to extend Fisher's discriminant for image data. The probability structure for our extended Fisher's discriminant is explicitly formulated, and the smoothness properties of images are utilized as prior probabilities. For the second question, we suggest a family of prior probabilities derived from natural image statistics. The unknown parameters in our model are estimated via the maximum a posteriori probability (MAP) estimation. We will show that existing methods imposing smoothness assumption of images are rough approximations to the proposed MAP estimates in this framework. Experimental results on the Yale face database and the ETH-80 object categorization dataset show that the proposed method significantly outperforms the other Fisher's discriminant methods for various image data.

[1]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[2]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[4]  J. Ramsay,et al.  Some Tools for Functional Data Analysis , 1991 .

[5]  Alejandro F. Frangi,et al.  Two-dimensional PCA: a new approach to appearance-based face representation and recognition , 2004 .

[6]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.

[7]  B. Silverman,et al.  Functional Data Analysis , 1997 .

[8]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[9]  Haiping Lu,et al.  Uncorrelated Multilinear Discriminant Analysis With Regularization and Aggregation for Tensor Object Recognition , 2009, IEEE Transactions on Neural Networks.

[10]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  J. S. Marron,et al.  Geometric representation of high dimension, low sample size data , 2005 .

[12]  Haiping Lu,et al.  Uncorrelated Multilinear Principal Component Analysis for Unsupervised Multilinear Subspace Learning , 2009, IEEE Transactions on Neural Networks.

[13]  David Mumford,et al.  Statistics of range images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.

[15]  B. Silverman,et al.  Canonical correlation analysis when the data are curves. , 1993 .

[16]  Yuxiao Hu,et al.  Learning a Spatially Smooth Subspace for Face Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Jieping Ye,et al.  Two-Dimensional Linear Discriminant Analysis , 2004, NIPS.

[18]  Bernt Schiele,et al.  Analyzing appearance and contour based methods for object categorization , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[20]  J. Ramsay.  When the data are functions , 1982 .

[21]  Ludwig Fahrmeir,et al.  Bayesian smoothing and regression for longitudinal, spatial and event history data / Ludwig Fahrmeir , 2011 .

[22]  Edward H. Adelson,et al.  Statistics of real-world illumination , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  T. Choi,et al.  Gaussian Process Regression Analysis for Functional Data , 2011 .

[25]  R. Tibshirani,et al.  Penalized Discriminant Analysis , 1995 .

[26]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..