Shadow removal of projected imagery by occluder shape measurement in a multiple overlapping projection system

This paper presents a shadow removal technique for a multiple overlapping projection system. In particular, this paper deals with situations where cameras cannot be placed between the occluder and projection surface. We apply a synthetic aperture capturing technique to estimate the appearance of the projection surface, and a visual hull reconstruction technique to measure the shape of the occluder. Once the shape is acquired, shadow regions on the surface can be estimated. The proposed shadow removal technique allows users to balance between the following two criteria: the likelihood of new shadow emergence and the spatial resolution of the projected results. Through a real projection experiment, we evaluate the proposed shadow removal technique

[1]  James M. Rehg,et al.  Shadow elimination and occluder light suppression for multi-projector displays , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[2]  Nassir Navab,et al.  Efficient visual hull computation for real-time 3D reconstruction using CUDA , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Greg Welch,et al.  Shader Lamps: Animating Real Objects With Image-Based Illumination , 2001, Rendering Techniques.

[4]  Gordon Wetzstein,et al.  The visual computing of projector-camera systems , 2008, SIGGRAPH '08.

[5]  Kosuke Sato,et al.  Dynamic defocus and occlusion compensation of projected imagery by model-based optimal projector selection in multi-projection environment , 2011, Virtual Reality.

[6]  Ramesh Raskar,et al.  Dynamic shader lamps : painting on movable objects , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[7]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Marc Levoy,et al.  Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  K. Sato,et al.  Range imaging system utilizing nematic liquid crystal mask , 1987 .

[10]  Anselmo Lastra,et al.  Life-sized projector-based dioramas , 2001, VRST '01.

[11]  H. M. Karara,et al.  Direct Linear Transformation from Comparator Coordinates into Object Space Coordinates in Close-Range Photogrammetry , 2015 .

[12]  Christopher O. Jaynes,et al.  Camera-based detection and removal of shadows from interactive multiprojector displays , 2004, IEEE Transactions on Visualization and Computer Graphics.

[13]  Oliver Bimber,et al.  Multifocal projection: a multiprojector technique for increasing focal depth , 2006, IEEE Transactions on Visualization and Computer Graphics.

[14]  Hideki Koike,et al.  Contrasting shadow for occluder light suppression from one-shot image , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[15]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[16]  W. Brent Seales,et al.  Dynamic shadow removal from front projection displays , 2001, Proceedings Visualization, 2001. VIS '01..

[17]  Brian P. Bailey,et al.  Build your world and play in it: Interacting with surface particles on complex objects , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[18]  Jeremy R. Cooperstock,et al.  Shadow Removal in Front Projection Environments Using Object Tracking , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Carl Machover,et al.  Virtual reality , 1994, IEEE Computer Graphics and Applications.

[20]  James M. Rehg,et al.  Shadow Elimination and Blinding Light Suppression for Interactive Projected Displays , 2007, IEEE Trans. Vis. Comput. Graph..

[21]  Gordon Wetzstein,et al.  The Visual Computing of Projector‐Camera Systems , 2008, SIGGRAPH '08.

[22]  T. Yoshida,et al.  A Virtual Color Reconstruction System for Real Heritage with Light Projection , 2003 .

[23]  Gordon Wetzstein,et al.  Enabling view-dependent stereoscopic projection in real environments , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[24]  Gita Reese Sukthankar,et al.  Dynamic shadow elimination for multi-projector displays , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.