ProDebNet: projector deblurring using a convolutional neural network.

Projection blur can occur in practical use cases that have non-planar and/or multi-projection display surfaces with various scattering characteristics because the surface often causes defocus and subsurface scattering. To address this issue, we propose ProDebNet, an end-to-end real-time projection deblurring network that synthesizes a projection image to minimize projection blur. The proposed method generates a projection image without explicitly estimating any geometry or scattering characteristics of the projection screen, which makes real-time processing possible. In addition, ProDebNet does not require real captured images for training data; we design a "pseudo-projected" synthetic dataset that is well-generalized to real-world blur data. Experimental results demonstrate that the proposed ProDebNet compensates for two dominant types of projection blur, i.e., defocus blur and subsurface blur, significantly faster than the baseline method, even in a real-projection scene.

[1]  Gordon Wetzstein,et al.  Coded aperture projection , 2008, SIGGRAPH '08.

[2]  Masatoshi Ishikawa,et al.  Dynamic Projection Mapping onto Deforming Non-Rigid Surface Using Deformable Dot Cluster Marker , 2017, IEEE Transactions on Visualization and Computer Graphics.

[3]  Bruno Lévy,et al.  Geometry-aware direction field processing , 2009, TOGS.

[4]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Kosuke Sato,et al.  Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture , 2015, Virtual Reality.

[6]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Shree K. Nayar,et al.  Projection defocus analysis for scene capture and image display , 2006, SIGGRAPH 2006.

[8]  Hans-Peter Seidel,et al.  Interactive by-example design of artistic packing layouts , 2013, ACM Trans. Graph..

[9]  Anselm Grundhöfer,et al.  Augmenting physical avatars using projector-based illumination , 2013, ACM Trans. Graph..

[10]  Hans-Peter Seidel,et al.  Design and volume optimization of space structures , 2017, ACM Trans. Graph..

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Hiroshi Ishikawa,et al.  Globally and locally consistent image completion , 2017, ACM Trans. Graph..