Multispectral Direct-Global Separation of Dynamic Scenes

In this paper, we propose a method for separating direct and global components of a dynamic scene per illumination color by using a projector-camera system; it exploits both the color switch and the temporal dithering of a DLP projector. Our proposed method is easy-to-implement because it does not require any self-built equipment and temporal synchronization between a projector and a camera. In addition, our method automatically calibrates the projector-camera correspondence in a dynamic scene on the basis of the consistency in pixel intensities, and optimizes the projection pattern on the basis of noise propagation analysis. We implemented the prototype setup and achieved multispectral direct-global separation of dynamic scenes in 60 Hz. Furthermore, we demonstrated that our method is effective for applications such as image-based material editing and multispectral relighting of dynamic scenes where wavelength-dependent phenomena such as fluorescence are observed.

[1]  Shuntaro Yamazaki,et al.  Temporal Dithering of Illumination for Fast Active Vision , 2008, ECCV.

[2]  Takahiro Okabe,et al.  Fast Spectral Reflectance Recovery Using DLP Projector , 2010, International Journal of Computer Vision.

[3]  Toshiyuki Amano.  Shading illusion: A novel way for 3-D representation on paper media , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[4]  Soheil Darabi,et al.  Compressive Dual Photography , 2009, Comput. Graph. Forum.

[5]  Greg Welch,et al.  Shader Lamps: Animating Real Objects With Image-Based Illumination , 2001, Rendering Techniques.

[6]  Imari Sato,et al.  Image-Based Separation of Reflective and Fluorescent Components Using Illumination Variant and Invariant Color , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Takahiro Okabe,et al.  Acquiring multispectral light transport using multi-primary DLP projector , 2016, 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA).

[8]  Pieter Peers,et al.  Compressive light transport sensing , 2009, ACM Trans. Graph..

[9]  Shree K. Nayar,et al.  A theory of multiplexed illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Paul E. Debevec,et al.  Acquiring the reflectance field of a human face , 2000, SIGGRAPH.

[11]  William H. Press,et al.  Numerical Recipes 3rd Edition: The Art of Scientific Computing , 2007 .

[12]  Kobus Barnard.  Color Constancy with Fluorescent Surfaces , 1999, Color Imaging Conference.

[13]  Zhouchen Lin,et al.  Kernel Nyström method for light transport , 2009, ACM Trans. Graph..

[14]  Shree K. Nayar,et al.  Multiplexed illumination for scene recovery in the presence of global illumination , 2011, 2011 International Conference on Computer Vision.

[15]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, SIGGRAPH 2006.

[16]  Yasushi Yagi,et al.  Analysis of light transport in scattering media , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Gordon Wetzstein,et al.  The Visual Computing of Projector‐Camera Systems , 2008, SIGGRAPH '08.