Riemannian manifold optimisation for non-rigid structure from motion

This paper address the problem of automatically extracting the 3D configurations of deformable objects from 2D features. Our focus in this work is to build on the observation that the subspace spanned by the motion parameters is a subset of a smooth manifold, and therefore we hunt for the solution in this space, rather than use heuristics (as previously attempted earlier). We succeed in this by attaching a canonical Riemannian metric, and using a variant of the non-rigid factorisation algorithm for structure from motion. We qualitatively and quantitatively show that our algorithm produces better results when compared to the state of art.

[1]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[2]  S. Shankar Sastry,et al.  Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation , 2001, International Journal of Computer Vision.

[3]  Marc Pollefeys,et al.  A Factorization-Based Approach to Articulated Motion Recovery , 2005, CVPR.

[4]  Henning Biermann,et al.  Recovering non-rigid 3D shape from image streams , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  Jing Xiao,et al.  A Closed-Form Solution to Non-rigid Shape and Motion Recovery , 2004, ECCV.

[6]  Matthew Brand,et al.  A direct method for 3D factorization of nonrigid motion observed in 2D , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Alessio Del Bue,et al.  Non-rigid structure from motion using ranklet-based tracking and non-linear optimization , 2007, Image Vis. Comput..

[8]  Jh Manton.  On the various generalisations of optimisation algorithms to manifold , 2004 .

[9]  Ian D. Reid,et al.  Articulated structure from motion by factorization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  P. Priouret,et al.  Newton's method on Riemannian manifolds: covariant alpha theory , 2002, math/0209096.

[11]  Harry Shum,et al.  Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Andrew W. Fitzgibbon,et al.  Damped Newton algorithms for matrix factorization with missing data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Takayuki Okatani,et al.  On the Wiberg Algorithm for Matrix Factorization in the Presence of Missing Components , 2007, International Journal of Computer Vision.

[14]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[15]  John M. Lee.  Introduction to Smooth Manifolds , 2002 .

[16]  Aaron Hertzmann,et al.  Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.