Image-based object reconstruction using run-length representation

This paper presents an image-based object reconstruction with a low memory footprint using run-length representation. While conventional volume-based approaches, which utilize voxels as primitives, are intuitive and easy to manipulate 3D data, they require a large amount of memory and computation during the reconstruction process. To overcome these burdens, this paper uses 3D runs to represent a 3D object and reconstructs each 3D run from multi-view silhouettes with a small amount of memory. The proposed geometry reconstruction is also computationally inexpensive, as it processes multiple voxels simultaneously. And for the compatibility with the conventional data formats, generation of polygonal 3D meshes from the reconstructed 3D runs is proposed as well. Lastly, texture mapping is proposed to additionally reduce the amount of memory for object reconstruction. The proposed reconstruction scheme has been simulated using various types of multi-view datasets. The results show that the proposed method performs object reconstruction with a smaller amount of memory and computation than voxel-based approaches. An image-based object reconstruction using run-length representation is proposed.A fast geometry reconstruction by rectifying images is proposed.A 3D mesh generation algorithm from the reconstructed 3D runs is proposed.View dependent texture mapping algorithm using a color palette is proposed.

[1]  Kun Li,et al.  Markerless Shape and Motion Capture From Multiview Video Sequences , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Takeo Kanade,et al.  Virtualized Reality: Perspectives on 4D Digitization of Dynamic Events , 2007, IEEE Computer Graphics and Applications.

[3]  Xiaojun Wu,et al.  Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video , 2004, Comput. Vis. Image Underst..

[4]  Bruno Raffin,et al.  A Distributed Approach for Real Time 3D Modeling , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[5]  Hans-Peter Seidel,et al.  A Flexible and Versatile Studio for Synchronized Multi-View Video Recording , 2003, VVG.

[6]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Adrian Hilton,et al.  Towards A 3D Virtual Studio forHuman Appearance Capture , 2003, VVG.

[8]  Ventseslav Sainov,et al.  3-D Time-Varying Scene Capture Technologies—A Survey , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Luc Van Gool,et al.  Markerless tracking of complex human motions from multiple views , 2006, Comput. Vis. Image Underst..

[10]  Seong-Dae Kim,et al.  Efficient three-dimensional object representation and reconstruction using depth and texture maps , 2008 .

[11]  Oliver Grau,et al.  A combined studio production system for 3-D capturing of live action and immersive actor feedback , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[13]  Hee-Dong Kim,et al.  High-resolution 3D Object Reconstruction using Multiple Cameras , 2013 .

[14]  Hans-Peter Seidel,et al.  Model-Based Free-Viewpoint Video Acquisition, Rendering and Encoding , 2004 .

[15]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[16]  Josep R. Casas,et al.  Multi-View Video Representation Based on Fast Monte Carlo Surface Reconstruction , 2013, IEEE Transactions on Image Processing.

[17]  Seong-Dae Kim,et al.  Geometric Properties of 3D Data Represented by DoSurface , 2012 .

[18]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[19]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Petros Daras,et al.  Real-Time, Full 3-D Reconstruction of Moving Foreground Objects From Multiple Consumer Depth Cameras , 2013, IEEE Transactions on Multimedia.

[21]  Xiaojun Wu,et al.  Real-time dynamic 3-D object shape reconstruction and high-fidelity texture mapping for 3-D video , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Markus H. Gross,et al.  3D video recorder , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[23]  Changsheng Xu,et al.  Mobile Landmark Search with 3D Models , 2014, IEEE Transactions on Multimedia.

[24]  Jérémie Allard,et al.  The GrImage Platform: A Mixed Reality Environment for Interactions , 2006, Fourth IEEE International Conference on Computer Vision Systems (ICVS'06).

[25]  Adrian Hilton,et al.  Surface Capture for Performance-Based Animation , 2007, IEEE Computer Graphics and Applications.

[26]  Adrian Hilton,et al.  Virtual view synthesis of people from multiple view video sequences , 2005, Graph. Model..

[27]  Laxmikant V. Kalé,et al.  A voxel-based parallel collision detection algorithm , 2002, ICS '02.

[28]  Michele Sanna,et al.  Enabling Geometry-Based 3-D Tele-Immersion With Fast Mesh Compression and Linear Rateless Coding , 2014, IEEE Transactions on Multimedia.

[29]  Adrian Hilton,et al.  Multiple view reconstruction of people , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[30]  Arie E. Kaufman,et al.  The voxblt Engine: A Voxel Frame Buffer Processor , 1988, Advances in Computer Graphics Hardware.

[31]  Seong-Dae Kim,et al.  3D binary morphological operations using run-length representation , 2008, Signal Process. Image Commun..

[32]  Luc Van Gool,et al.  Blue-c: a spatially immersive display and 3D video portal for telepresence , 2003, IPT/EGVE.

[33]  Takashi Matsuyama,et al.  Cell-Based 3D Video Capture Method with Active Cameras , 2010, Image and Geometry Processing for 3-D Cinematography.

[34]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[35]  Seong-Dae Kim,et al.  Fast Computation of a Visual Hull , 2010, ACCV.

[36]  T. Matsuyama,et al.  Dynamic 3D shape from multi-viewpoint images using deformable mesh model , 2003, 3rd International Symposium on Image and Signal Processing and Analysis, 2003. ISPA 2003. Proceedings of the.

[37]  Markus H. Gross,et al.  3D video fragments: dynamic point samples for real-time free-viewpoint video , 2004, Comput. Graph..

[38]  Yuichi Iwadate,et al.  Algorithm for dynamic 3D object generation from multi-viewpoint images , 2004, SPIE Optics East.

[39]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[40]  Yunhui Liu,et al.  Sphere Image for 3-D Model Retrieval , 2014, IEEE Transactions on Multimedia.

[41]  Adrian Hilton,et al.  The Multiple-Camera 3-D Production Studio , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Mohan M. Trivedi,et al.  Human Body Model Acquisition and Tracking Using Voxel Data , 2003, International Journal of Computer Vision.

[43]  Takeo Kanade,et al.  Image-based spatio-temporal modeling and view interpolation of dynamic events , 2005, TOGS.

[44]  Edmond Boyer,et al.  Exact polyhedral visual hulls , 2003, BMVC.

[45]  Takashi Matsuyama,et al.  3D Video and Its Applications , 2012, Springer London.

[46]  Hans-Peter Seidel,et al.  Performance capture from sparse multi-view video , 2008, ACM Trans. Graph..

[47]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[48]  S. Goldsack,et al.  IN REAL-TIME , 2008 .

[49]  Horst Bischof,et al.  Rapid Skin: Estimating the 3D Human Pose and Shape in Real-Time , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[50]  Adrian Hilton,et al.  Model-based multiple view reconstruction of people , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[51]  G. Ziegler,et al.  High-Quality Reconstruction from Multiview Video Streams , 2007, IEEE Signal Processing Magazine.

[52]  Larry S. Davis,et al.  A distributed system for real-time volume reconstruction , 2000, Proceedings Fifth IEEE International Workshop on Computer Architectures for Machine Perception.

[53]  Daniel Cohen-Or,et al.  Volume graphics , 1993, Computer.