Extracting Multi-scalc Structure from Data

Sparse Principal Component Analysis (S-PCA) is a novel framework for learning a linear, orthonormal basis repre- sentation for structure intrinsic to an ensemble of images. S-PCA is based on the discovery that natural images ex- hibit structure in a low-dimensional subspace in a sparse, scale-dependent form. The S-PCA basis optimizes an ob- jective function which trades off correlations among output coeficients for sparsity in the description of basis vector el- ements. This objective function is minimized by a simple, robust and highly scalable adaptation algorithm, consisting of successive planar rotations of pairs of basis vectors. The formulation of S-PCA is novel in that multi-scale represen- tations emerge for a variety of ensembles including face im- ages, images from outdoor scenes and a database of optical jlow vectors representing a motion class.