Fast Transformation-Invariant Component Analysis

Abstract Dimensionality reduction techniques such as principal component analysis and factor analysis are used to discover a linear mapping between high-dimensional data samples and points in a lower-dimensional subspace. Previously, Frey and Jojic introduced transformation-invariant component analysis (TCA) to learn a linear mapping, invariant to a set of known form of global transformations. However, parameter estimation in that model using the previously-proposed expectation maximization (EM) algorithm required scalar operations in the order of N2 where N is the dimensionality of each training example. This is prohibitive for many applications of interest such as modeling mid-to large-size images, where, for instance, N may be as high as 786432 (512×512 RGB image). In this paper, we present an efficient algorithm that reduces the computational requirements to order of Nlog N. With this speedup, we show the effectiveness of transformation-invariant component analysis in various applications including tracking, learning video textures, clustering, object recognition and object detection in images. Software for TCA can be downloaded from http://www.psi.toronto.edu/fastTCA.htm.

[1]  Arnold Neumaier,et al.  Estimation of parameters and eigenmodes of multivariate autoregressive models , 2001, TOMS.

[2]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[3]  William H. Press,et al.  Numerical recipes in C , 2002 .

[4]  George Wolberg,et al.  Robust image registration using log-polar transform , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[5]  Larry S. Davis,et al.  Improved fast gauss transform and efficient kernel density estimation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  Brendan J. Frey,et al.  Estimating mixture models of images and inferring spatial transformations using the EM algorithm , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[7]  Stan Z. Li,et al.  Learning to detect multi-view faces in real-time , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[8]  Andrew W. Moore,et al.  'N-Body' Problems in Statistical Learning , 2000, NIPS.

[9]  Brendan J. Frey,et al.  Learning appearance and transparency manifolds of occluded objects in layers , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Michael J. Black,et al.  EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation , 1996, International Journal of Computer Vision.

[11]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[12]  Erik G. Learned-Miller,et al.  Data driven image models through continuous joint alignment , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Jon M. Kleinberg,et al.  Fast Algorithms for Large-State-Space HMMs with Applications to Web Usage Analysis , 2003, NIPS.

[14]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Brendan J. Frey,et al.  Fast Transformation-Invariant Factor Analysis , 2002, NIPS.

[16]  Padhraic Smyth,et al.  Probabilistic Models For Joint Clustering And Time-Warping Of Multidimensional Curves , 2003, UAI.

[17]  Brendan J. Frey,et al.  Transformed component analysis: joint estimation of spatial transformations and image components , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Brendan J. Frey,et al.  Fast, Large-Scale Transformation-Invariant Clustering , 2001, NIPS.

[19]  Geoffrey E. Hinton,et al.  The EM algorithm for mixtures of factor analyzers , 1996 .

[20]  Andrew S. Glassner,et al.  Proceedings of the 27th annual conference on Computer graphics and interactive techniques , 1994, SIGGRAPH 1994.

[21]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[22]  Brian Everitt,et al.  An Introduction to Latent Variable Models , 1984 .

[23]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[24]  David J. Fleet,et al.  A framework for modeling appearance change in image sequences , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[25]  B. Frey,et al.  Transformation-Invariant Clustering Using the EM Algorithm , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Brendan J. Frey,et al.  Mixtures of local linear subspaces for face recognition , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).