Joint estimation of shape and motion from silhouettes

This paper presents a fast algorithm for recovering shape and motion of dynamic scenes from sequences of silhouettes in calibrated multi-camera settings in wide-baseline configurations. The proposed method captures the static shape on the first frame by deforming an arbitrary set of points located on an initial surface enclosing the actual scene. Then, the initial 3D shape is propagated in subsequent frames by quickly deforming the initial static shape in order to fit the available silhouettes. The resulting points lie on the surface of the Visual Hull. Shape and motion descriptions are jointly obtained by the dynamic procedure that adapts existing surfaces to the set of silhouettes at each new time instant. Experimental results show how the proposed dynamic method obtains silhouette-consistent shape estimates -with the same accuracy than a static approach- and dense motion estimates with very low usage of computational resources, by exploiting both spatial and temporal redundancies.

[1]  Jean-Philippe Pons,et al.  Dense and Accurate Spatio-temporal Multi-view Stereovision , 2009, ACCV.

[2]  Adrian Hilton,et al.  Spherical matching for temporal correspondence of non-rigid surfaces , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[3]  Michael M. Kazhdan,et al.  Poisson surface reconstruction , 2006, SGP '06.

[4]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[5]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Marc Pollefeys,et al.  Multi-object shape estimation and tracking from silhouette cues , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Marcus A. Magnor,et al.  Space-time isosurface evolution for temporally coherent 3D reconstruction , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[8]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  Touradj Ebrahimi,et al.  MESH: measuring errors between surfaces using the Hausdorff distance , 2002, Proceedings. IEEE International Conference on Multimedia and Expo.

[10]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.

[11]  Edmond Boyer,et al.  Efficient Polyhedral Modeling from Silhouettes , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Ventseslav Sainov,et al.  3-D Time-Varying Scene Capture Technologies—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Jean Ponce,et al.  Dense 3D motion capture from synchronized video streams , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Hans-Peter Seidel,et al.  Marker-less Deformable Mesh Tracking for Human Shape and Motion Capture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.