Exploiting DLP Illumination Dithering for Reconstruction and Photography of High-Speed Scenes

In this work, we recover fast moving scenes by exploiting the high-speed illumination “dithering” of cheap and easily available digital light processing (DLP) projectors. We first show how to reverse-engineer the temporal dithering for off-the-shelf projectors, using a high-speed camera. DLP dithering can produce temporal patterns commonly used in active vision techniques. Since the dithering occurs at a very high frame-rate, such illumination-based methods can be “speed up” for fast scenes. We demonstrate this with three applications, each of which only requires a single slide to be displayed by the DLP projector. The quality of the result is determined by the camera frame-rate available to the user. Pairing a high-speed camera and a DLP projector, we demonstrate structured light reconstruction at 100 Hz. With the same camera and three or more DLP projectors, we show photometric stereo and demultiplexing applications at 300 Hz. Finally, with a real-time (60 Hz) or still camera, we show that DLP illumination acts as a very fast flash, allowing strobe photography of high-speed scenes. We discuss, in depth, some characteristics of the temporal dithering with a case study of a particular projector. Finally, we describe limitations, trade-offs and other issues relating to this work.

[1]  Paul J. Besl,et al.  Active, optical range imaging sensors , 1988, Machine Vision and Applications.

[2]  Hideshi Yamada,et al.  Rendering for an interactive 360° light field display , 2007, ACM Trans. Graph..

[3]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[4]  Srinivasa G. Narasimhan,et al.  Illustrating motion through DLP photography , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[5]  Takashi Shibata,et al.  Ergonomic evaluation of a field-sequential colour projection system , 2008, Displays.

[6]  Shuntaro Yamazaki,et al.  Temporal Dithering of Illumination for Fast Active Vision , 2008, ECCV.

[7]  Shojiro Sakata,et al.  Reconstruction Of Surfaces Of 3-D Objects By M-array Pattern Projection Method , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[8]  Mostafa Kaveh,et al.  A regularization approach to joint blur identification and image restoration , 1996, IEEE Trans. Image Process..

[9]  Jeffrey L. Posdamer,et al.  Surface measurement by space-encoded projected beam systems , 1982, Comput. Graph. Image Process..

[10]  Joel Pokorny,et al.  Characterization and use of a digital light projector for vision research , 2001, Vision Research.

[11]  Toni Järvenpää,et al.  Measuring color breakup of stationary images in field‐sequential‐color displays , 2005 .

[12]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[13]  Takeo Miyasaka,et al.  HIGH SPEED 3-D MEASUREMENT SYSTEM USING INCOHERENT LIGHT SOURCE FOR HUMAN PERFORMANCE ANALYSIS , 2000 .

[14]  Luc Van Gool,et al.  Fast 3D Scanning with Automatic Motion Compensation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Steve Marschner,et al.  Dual photography , 2005, ACM Trans. Graph..

[16]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Scott E. Hudson,et al.  Moveable interactive projected displays using projector based tracking , 2005, UIST.

[18]  Masahiko Inami,et al.  Smart Light-Ultra High Speed Projector for Spatial Multiplexing Optical Transmission , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[19]  Szymon Rusinkiewicz,et al.  Spacetime Stereo: A Unifying Framework for Depth from Triangulation , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  K. Ukai,et al.  Visual fatigue in congenital nystagmus caused by viewing images of color sequential projectors , 2005, Journal of Display Technology.

[21]  Takashi Shibata,et al.  Ergonomic Evaluation of a Projector using Field Sequential Color Projection System , 2004 .

[22]  William T. Freeman,et al.  Shape-time photography , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Paul A. Beardsley,et al.  A self-correcting projector , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[24]  Edward M. Reingold,et al.  Efficient generation of the binary reflected gray code and its applications , 1976, CACM.

[25]  Andrew Gardner,et al.  Performance relighting and reflectance transformation with time-multiplexed illumination , 2005, ACM Trans. Graph..

[26]  Joaquim Salvi,et al.  Pattern codification strategies in structured light systems , 2004, Pattern Recognit..

[27]  Kim L. Boyer,et al.  Color-Encoded Structured Light for Rapid Active Ranging , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Walter M. Duncan,et al.  Emerging digital micromirror device (DMD) applications , 2003, SPIE MOEMS-MEMS.

[29]  Shree K. Nayar,et al.  A theory of multiplexed illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[30]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[31]  Marc Levoy,et al.  Better optical triangulation through spacetime analysis , 1995, Proceedings of IEEE International Conference on Computer Vision.

[32]  Shree K. Nayar,et al.  Programmable imaging using a digital micromirror array , 2004, CVPR 2004.

[33]  Peter M. Will,et al.  Grid Coding: A Preprocessing Technique for Robot and Machine Vision , 1971, IJCAI.

[34]  Nahum Kiryati,et al.  Toward optimal structured light patterns , 1997, Proceedings. International Conference on Recent Advances in 3-D Digital Imaging and Modeling (Cat. No.97TB100134).

[35]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[37]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH 2006.

[38]  Shree K. Nayar,et al.  Projection defocus analysis for scene capture and image display , 2006, SIGGRAPH 2006.

[39]  K. Sato,et al.  Range imaging system utilizing nematic liquid crystal mask , 1987 .

[40]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[41]  Qian Chen,et al.  A light modulation/demodulation method for real-time 3D imaging , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[42]  Mineo Mori,et al.  Mechanism of color breakup in field‐sequential‐color projectors , 1999 .

[43]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[45]  Steven M. Seitz,et al.  Shape and materials by example: a photometric stereo approach , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[46]  Marc Levoy,et al.  Dual photography , 2005, SIGGRAPH 2005.

[47]  Szymon Rusinkiewicz,et al.  Viewpoint-Coded Structured Light , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Ramesh Raskar,et al.  Fast separation of direct and global components of a scene using high frequency illumination , 2006, ACM Trans. Graph..

[49]  Richard G. Baraniuk,et al.  A new compressive imaging camera architecture using optical-domain compression , 2006, Electronic Imaging.