Stochastic Search for Optimal Linear Representations of Images on Spaces with Orthogonality Constraints

Simplicity of linear representations makes them a popular tool in several imaging analysis, and indeed many other applications involving high-dimensional data. In image analysis, the two widely used linear representations are: (i) linear projections of images to low-dimensional Euclidean subspaces, and (ii) linear spectral filtering of images. In view of the orthogonality and other constraints imposed on these representations (the subspaces or the filters), they take values on nonlinear manifolds (Grassmann, Stiefel, or rotation group). We present a family of algorithms that exploit the geometry of the underlying manifolds to find optimal linear representations for specified tasks. We illustrate the effectiveness of algorithms by finding optimal subspaces and sparse filters both in the context of image-based object recognition.

[1]  F. W. Warner.  Foundations of Differentiable Manifolds and Lie Groups , 1971 .

[2]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[3]  W. Boothby.  An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[4]  U. Grenander,et al.  Ergodic Algorithms on Special Euclidean Groups for ATR , 1997 .

[5]  S. Geman,et al.  Diffusions for global optimizations , 1986 .

[6]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[7]  Michael I. Miller,et al.  REPRESENTATIONS OF KNOWLEDGE IN COMPLEX SYSTEMS , 1994 .

[8]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[9]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[10]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[11]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[12]  U. Grenander,et al.  Jump–diffusion Markov processes on orthogonal groups for object pose estimation , 2002 .

[13]  Hiroshi Murase,et al.  Visual learning and recognition of 3-d objects from appearance , 2005, International Journal of Computer Vision.

[14]  Y. Amit.  A multiflow approximation to diffusions , 1991 .

[15]  Xiuwen Liu,et al.  Independent spectral representations of images for recognition. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[16]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[17]  Aapo Hyvärinen,et al.  The Fixed-Point Algorithm and Maximum Likelihood Estimation for Independent Component Analysis , 1999, Neural Processing Letters.

[18]  Song-Chun Zhu,et al.  Minimax Entropy Principle and Its Application to Texture Modeling , 1997, Neural Computation.