3D Reconstruction by Shadow Carving: Theory and Practical Evaluation

Cast shadows are an informative cue to the shape of objects. They are particularly valuable for discovering object’s concavities which are not available from other cues such as occluding boundaries. We propose a new method for recovering shape from shadows which we call shadow carving. Given a conservative estimate of the volume occupied by an object, it is possible to identify and carve away regions of this volume that are inconsistent with the observed pattern of shadows. We prove a theorem that guarantees that when these regions are carved away from the shape, the shape still remains conservative. Shadow carving overcomes limitations of previous studies on shape from shadows because it is robust with respect to errors in shadows detection and it allows the reconstruction of objects in the round, rather than just bas-reliefs. We propose a reconstruction system to recover shape from silhouettes and shadow carving. The silhouettes are used to reconstruct the initial conservative estimate of the object’s shape and shadow carving is used to carve out the concavities. We have simulated our reconstruction system with a commercial rendering package to explore the design parameters and assess the accuracy of the reconstruction. We have also implemented our reconstruction scheme in a table-top system and present the results of scanning of several objects.

[1]  Gregory Dudek,et al.  On 3-D surface reconstruction using shape from shadows , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[2]  Jean Ponce,et al.  On computing exact visual hulls of solids bounded by smooth surfaces , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Peter Eisert,et al.  Multi-hypothesis, volumetric reconstruction of 3-D objects from multiple calibrated camera views , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[4]  P. Perona,et al.  Shadow carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Takeo Kanade,et al.  Using shadows in finding surface orientations , 1983, Comput. Vis. Graph. Image Process..

[6]  Richard S. Weiss,et al.  Reconstruction of Surfaces from Profiles , 1987, ICCV 1987.

[7]  Mark R. Stevens,et al.  Methods for Volumetric Reconstruction of Visual Scenes , 2004, International Journal of Computer Vision.

[8]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[9]  W. RICHARD STARK.  Automatic model construction , 1980, Inf. Sci..

[10]  Marie-Odile Berger,et al.  3D Surface Reconstruction Using Occluding Contours , 2004, International Journal of Computer Vision.

[11]  Pietro Perona,et al.  Implementation of a shadow carving system for shape capture , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[12]  Richard Szeliski,et al.  Stereo Matching with Transparency and Matting , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  Kenneth A. Loparo,et al.  Reconstruction of three-dimensional surfaces from two-dimensional binary images , 1989, IEEE Trans. Robotics Autom..

[14]  David J. Kriegman,et al.  What Shadows Reveal about Object Structure , 1998, ECCV.

[15]  Adrien Treuille,et al.  Example-Based Stereo with General BRDFs , 2004, ECCV.

[16]  Andrew W. Fitzgibbon,et al.  Automatic 3D Model Construction for Turn-Table Sequences , 1998, SMILE.

[17]  J J Koenderink,et al.  What Does the Occluding Contour Tell Us about Solid Shape? , 1984, Perception.

[18]  R. Cipolla,et al.  A probabilistic framework for space carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[19]  David J. Kriegman,et al.  What shadows reveal about object structure. , 2001 .

[20]  Jean Ponce,et al.  Automatic model construction, pose estimation, and object recognition from photographs using triangular splines , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[21]  Yizhou Yu,et al.  Shadow Graphs and 3D Texture Reconstruction , 2005, Int. J. Comput. Vis..

[22]  Richard Szeliski,et al.  Rapid octree construction from image sequences , 1993 .

[23]  Gabriel Taubin,et al.  A signal processing approach to fair surface design , 1995, SIGGRAPH.

[24]  Jean Ponce,et al.  Automatic Model Construction and Pose Estimation From Photographs Using Triangular Splines , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Paul A. Viola,et al.  Roxels: responsibility weighted 3D volume reconstruction , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[26]  Pietro Perona,et al.  3D Photography Using Shadows in Dual-Space Geometry , 1999, International Journal of Computer Vision.

[27]  Johnny T. Chang,et al.  Shadow Graphs and 3D Texture Reconstruction , 2004, International Journal of Computer Vision.

[28]  John R. Kender,et al.  An optimal algorithm for the derivation of shape from shadows , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  William Ribarsky,et al.  Toward Spontaneous Interaction with the Perceptive Workbench , 2000, IEEE Computer Graphics and Applications.

[30]  Tung Hsu,et al.  Reconstruction of the surfaces , 1991 .

[31]  Roberto Cipolla,et al.  Structure and motion from silhouettes , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[32]  Gene H. Golub,et al.  Optimal Surface Smoothing as Filter Design , 1996, ECCV.

[33]  Paolo Cignoni,et al.  A low cost 3D scanner based on structured light , 2001 .

[34]  Murray H. Loew,et al.  The Entry-Exit Method of Shadow Boundary Segmentation , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Gregory Dudek,et al.  Space occupancy using multiple shadowimages , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[36]  Richard Szeliski,et al.  Stereo Matching with Transparency and Matting , 1999, International Journal of Computer Vision.

[37]  Steven M. Seitz,et al.  Photorealistic Scene Reconstruction by Voxel Coloring , 1997, International Journal of Computer Vision.

[38]  Olivier D. Faugeras,et al.  Using Extremal Boundaries for 3-D Object Modeling , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Guido M. Cortelazzo,et al.  Automatic 3D modeling of textured cultural heritage objects , 2004, IEEE Transactions on Image Processing.

[40]  Peter K. Allen,et al.  3-D Modeling from Range Imagery: An Incremental Method with a Planning Component , 1999, Image Vis. Comput..

[41]  Jiang Yu Zheng,et al.  Acquiring 3-D Models from Sequences of Contours , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[43]  Jake K. Aggarwal,et al.  Volumetric Descriptions of Objects from Multiple Views , 1983, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Martin Kampel,et al.  Octree-based fusion of shape from silhouette and shape from structured light , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[45]  Andrew Blake,et al.  Surface shape from the deformation of apparent contours , 1992, International Journal of Computer Vision.

[46]  Aldo Laurentini,et al.  How Far 3D Shapes Can Be Understood from 2D Silhouettes , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Paolo Cignoni,et al.  A low cost optical 3D scanner , 2001, Comput. Graph. Forum.

[48]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[49]  Kiriakos N. Kutulakos,et al.  A Probabilistic Theory of Occupancy and Emptiness , 2002, ECCV.

[50]  J. Kender,et al.  Shape from darkness under error , 1996 .

[51]  William E. Lorensen,et al.  Marching cubes: a high resolution 3D surface construction algorithm , 1996 .

[52]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[53]  SavareseSilvio,et al.  3D Reconstruction by Shadow Carving , 2007 .

[54]  Hisham El-Shishiny,et al.  Scanning and processing 3D objects for Web display , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[55]  Roger Y. Tsai,et al.  A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses , 1987, IEEE J. Robotics Autom..