Efficient multi-viewpoint acquisition of 3D objects undergoing repetitive motions

Computer graphics applications such as movie effects, video gaming, and product demonstration demand 3D models of dynamic objects. For this purpose, numerous methods, such as light fields, stereo reconstruction and visual hulls have been extended to model dynamic objects. These methods use multiple cameras to acquire images simultaneously and use the synchronized samples to reconstruct the model for each time instance. However, a large number of cameras are required to obtain compelling results. We introduce an efficient acquisition and modeling schema for dynamic objects with repetitive motions. Our method requires as few as two cameras. The key idea is that repetitive motions can be described by a finite number of states. Images capturing the same state can be grouped together and fed to the later modeling phase as if they are captured from multiple cameras simultaneously. Our work includes an acquisition system with interactive feedback, a graph traversal algorithm to help obtain a near minimum subset of images to sample the object and its motion, and a space-time image optimization method. We demonstrate this system using several datasets with different complexity of motion, and different number of desired viewpoints.

[1]  Steven M. Seitz,et al.  View-Invariant Analysis of Cyclic Motion , 1997, International Journal of Computer Vision.

[2]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[3]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH Classes.

[4]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[5]  Patrick Pérez,et al.  Periodic motion detection and segmentation via approximate sequence alignment , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[7]  Takeshi Naemura,et al.  Real-Time Video-Based Modeling and Rendering of 3D Scenes , 2002, IEEE Computer Graphics and Applications.

[8]  Adrian Hilton,et al.  Video-based character animation , 2005, SCA '05.

[9]  Leonard McMillan,et al.  Plenoptic Modeling: An Image-Based Rendering System , 2023 .

[10]  Andrew Zisserman,et al.  Multiple View Geometry , 1999 .

[11]  Andrew Jones,et al.  Relighting human locomotion with flowed reflectance fields , 2006, EGSR '06.

[12]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[13]  David Salesin,et al.  Panoramic video textures , 2005, ACM Trans. Graph..

[14]  David Salesin,et al.  Panoramic video textures , 2005, SIGGRAPH 2005.

[15]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[16]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[17]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[18]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.

[19]  Charles R. Dyer,et al.  Cyclic motion detection using spatiotemporal surfaces and curves , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[20]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[21]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[22]  Leonard McMillan,et al.  A Real-Time Distributed Light Field Camera , 2002, Rendering Techniques.

[23]  Fang Liu,et al.  Finding periodicity in space and time , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[24]  Huamin Wang,et al.  Towards space: time light field rendering , 2005, I3D '05.

[25]  Lucas Kovar,et al.  Motion Graphs , 2002, ACM Trans. Graph..

[26]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[27]  Marc Pollefeys,et al.  Automatic Kinematic Chain Building from Feature Trajectories of Articulated Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[28]  Mark R. Stevens,et al.  Methods for Volumetric Reconstruction of Visual Scenes , 2004, International Journal of Computer Vision.

[29]  Ruigang Yang,et al.  Real-time consensus-based scene reconstruction using commodity graphics hardware , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[30]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.