Hierarchical structure and nonrigid motion recovery from 2D monocular views

Inferring both 3D structure and motion of nonrigid objects from monocular images is an important problem in computer vision. The challenges stem not only from the absence of point correspondences but also from the structure ambiguity. In this paper, a hierarchical method which integrates both local patch analysis and global shape descriptions is devised to solve the dual problem of structure and nonrigid motion recovery by using an elastic geometric model-extended superquadrics (ESQ). The nonrigid object of interest is segmented into many small areas and local analysis is performed to recover small details for each small area, assuming that each small area is undergoing similar nonrigid motion. Then, a recursive algorithm is proposed to guide and regularize local analysis with global information by using an appropriate global ESQ model. This local-global hierarchy enables us to capture both local and global deformations accurately and robustly. Experimental results on both simulation and real data are presented to validate and evaluate the effectiveness and robustness of the proposed approach.