Flexible Images: Matching and Recognition Using Learned Deformations

We describe a novel technique for matching and recognition based on deformable intensity surfaces which incorporates both the shape (x,y) and the texture (I(x,y)) components of a 2D image. Specifically, the intensity surface is modeled as a deformable 3D mesh in (x,y,I(x,y)) space which obeys Lagrangian dynamics. Using an efficient technique for matching two surfaces (in terms of the analytic modes of vibration), we can obtain a dense correspondence field (or 3D warp) between two images. Furthermore, we use explicit statistical learning of the class of valid deformations in order to provide a priori knowledge about object-specific deformations. The resulting formulation leads to a compact representation based on the physically-based modes of deformation as well as the statistical modes of variation observed in actual training data. We demonstrate the power of this approach with experiments utilizing image matching, interpolation of missing data, and image retrieval in a large face database.