Modeling repetitive motions using structured light.

Obtaining models of dynamic 3D objects is an important part of content generation for computer graphics. Numerous methods have been extended from static scenarios to model dynamic scenes. If the states or poses of the dynamic object repeat often during a sequence (but not necessarily periodically), we call such a repetitive motion. There are many objects, such as toys, machines, and humans, undergoing repetitive motions. Our key observation is that when a motion-state repeats, we can sample the scene under the same motion state again but using a different set of parameters; thus, providing more information of each motion state. This enables robustly acquiring dense 3D information difficult for objects with repetitive motions using only simple hardware. After the motion sequence, we group temporally disjoint observations of the same motion state together and produce a smooth space-time reconstruction of the scene. Effectively, the dynamic scene modeling problem is converted to a series of static scene reconstructions, which are easier to tackle. The varying sampling parameters can be, for example, structured-light patterns, illumination directions, and viewpoints resulting in different modeling techniques. Based on this observation, we present an image-based motion-state framework and demonstrate our paradigm using either a synchronized or an unsynchronized structured-light acquisition method.

[1]  Philip Fong,et al.  High-resolution three-dimensional sensing of fast deforming objects , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Patrick Pérez,et al.  Periodic motion detection and segmentation via approximate sequence alignment , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[3]  Jovan Popovic,et al.  Continuous capture of skin deformation , 2003, ACM Trans. Graph..

[4]  Joseph Shamir,et al.  Range Imaging With Adaptive Color Structured Light , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Luc Van Gool,et al.  Real-time range scanning of deformable surfaces by adaptively coded structured light , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[6]  Paolo Cignoni,et al.  A low cost 3D scanner based on structured light , 2001 .

[7]  Tim Weyrich,et al.  A practical structured light acquisition system for point-based geometry and texture , 2005, Proceedings Eurographics/IEEE VGTC Symposium Point-Based Graphics, 2005..

[8]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[9]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.

[10]  Markus H. Gross,et al.  Scalable 3D video of dynamic scenes , 2005, The Visual Computer.

[11]  Daniel G. Aliaga,et al.  Dense Depth and Color Acquisition of Repetitive Motions , 2007, Sixth International Conference on 3-D Digital Imaging and Modeling (3DIM 2007).

[12]  Björn Stenger,et al.  Non-rigid Photometric Stereo with Colored Lights , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[13]  Luc Van Gool,et al.  Scene-adapted structured light , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  David Salesin,et al.  Panoramic video textures , 2005, ACM Trans. Graph..

[15]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[16]  KwangYun Wohn,et al.  The control of avatar motion using hand gesture , 1998, VRST '98.

[17]  Adrian Hilton,et al.  Video-based character animation , 2005, SCA '05.

[18]  Bobby Bodenheimer,et al.  Synthesis and evaluation of linear motion transitions , 2008, TOGS.

[19]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[20]  Derek Bradley,et al.  Markerless garment capture , 2008, ACM Trans. Graph..

[21]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[22]  Luc Van Gool,et al.  A graph cut based adaptive structured light approach for real-time range acquisition , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[23]  Lucas Kovar,et al.  Motion graphs , 2002, SIGGRAPH '08.

[24]  Andrew Gardner,et al.  Simulating spatially varying lighting on a live performance , 2006 .

[25]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[26]  S. Inokuchi,et al.  Range-imaging system for 3-D object recognition , 1984 .

[27]  Daniel G. Aliaga,et al.  Efficient multi-viewpoint acquisition of 3D objects undergoing repetitive motions , 2007, SI3D.

[28]  Takeo Kanade,et al.  A multibaseline stereo system with active illumination and real-time image acquisition , 1995, Proceedings of IEEE International Conference on Computer Vision.

[29]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[30]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[31]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[33]  Andrew Jones,et al.  Relighting human locomotion with flowed reflectance fields , 2006, EGSR '06.

[34]  Li Zhang,et al.  Spacetime faces: high resolution capture for modeling and animation , 2004, SIGGRAPH 2004.

[35]  Szymon Rusinkiewicz,et al.  Spacetime stereo: a unifying framework for depth from triangulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[37]  Christian Früh,et al.  Capturing 2 1/2 D depth and texture of time-varying scenes using structured infrared light , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[38]  Michael Bosse,et al.  Unstructured lumigraph rendering , 2001, SIGGRAPH.

[39]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[40]  Christian Früh,et al.  Capturing 2½D Depth and Texture of Time-Varying Scenes Using Structured Infrared Light , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[41]  Steven M. Seitz,et al.  View-Invariant Analysis of Cyclic Motion , 1997, International Journal of Computer Vision.