Shadow matting and compositing

In this paper, we describe a method for extracting shadows from one natural scene and inserting them into another. We develop physically-based shadow matting and compositing equations and use these to pull a shadow matte from a source scene in which the shadow is cast onto an arbitrary planar background. We then acquire the photometric and geometric properties of the target scene by sweeping oriented linear shadows (cast by a straight object) across it. From these shadow scans, we can construct a shadow displacement map without requiring camera or light source calibration. This map can then be used to deform the original shadow matte. We demonstrate our approach for both indoor scenes with controlled lighting and for outdoor scenes using natural lighting.

[1]  Steve Wright.  Digital Compositing for Film and Video with Cdrom , 2001 .

[2]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[3]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[4]  David Salesin,et al.  Video matting of complex scenes , 2002, SIGGRAPH.

[5]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  P. Anandan,et al.  Direct recovery of shape from multiple views: a parallax based approach , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[7]  Donald P. Greenberg,et al.  A user interface for interactive cinematic shadow design , 2002, SIGGRAPH.

[8]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[9]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[10]  P. Perona,et al.  Shadow carving , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[11]  Stephen Lin,et al.  Lighting interpolation by shadow morphing using intrinsic lumigraphs , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[12]  David Salesin,et al.  Environment matting extensions: towards higher accuracy and real-time capture , 2000, SIGGRAPH.

[13]  Carlo Tomasi,et al.  Alpha estimation in natural images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[16]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Pietro Perona,et al.  3D photography on your desk , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[19]  Adam Finkelstein,et al.  Shadows for cel animation , 2000, SIGGRAPH.

[20]  David Salesin,et al.  Environment matting and compositing , 1999, SIGGRAPH.

[21]  Steve Wright.  Digital Compositing for Film and Video , 2001 .

[22]  Tom Duff,et al.  Compositing digital images , 1984, SIGGRAPH.