DisCo: Display-Camera Communication Using Rolling Shutter Sensors

We present DisCo, a novel display-camera communication system. DisCo enables displays and cameras to communicate with each other while also displaying and capturing images for human consumption. Messages are transmitted by temporally modulating the display brightness at high frequencies so that they are imperceptible to humans. Messages are received by a rolling shutter camera that converts the temporally modulated incident light into a spatial flicker pattern. In the captured image, the flicker pattern is superimposed on the pattern shown on the display. The flicker and the display pattern are separated by capturing two images with different exposures. The proposed system performs robustly in challenging real-world situations such as occlusion, variable display size, defocus blur, perspective distortion, and camera rotation. Unlike several existing visible light communication methods, DisCo works with off-the-shelf image sensors. It is compatible with a variety of sources (including displays, single LEDs), as well as reflective surfaces illuminated with light sources. We have built hardware prototypes that demonstrate DisCo’s performance in several scenarios. Because of its robustness, speed, ease of use, and generality, DisCo can be widely deployed in several applications, such as advertising, pairing of displays with cell phones, tagging objects in stores and museums, and indoor navigation.

[1]  Kevin Curran,et al.  Digital image steganography: Survey and analysis of current methods , 2010, Signal Process..

[2]  S. Yoshimura,et al.  A 48 kframe/s CMOS image sensor for real-time 3-D sensing and motion detection , 2001, 2001 IEEE International Solid-State Circuits Conference. Digest of Technical Papers. ISSCC (Cat. No.01CH37177).

[3]  J. Tanida,et al.  Dynamic Reconfiguration of Differential Pixel Output for CMOS Imager Dedicated to WDM-SDM Indoor Optical Wireless LAN , 2009, IEEE Photonics Technology Letters.

[4]  Ramesh Raskar,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, SIGGRAPH '06.

[5]  Ramjee Prasad,et al.  OFDM for Wireless Multimedia Communications , 1999 .

[6]  Per Ola Börjesson,et al.  ML estimation of time and frequency offset in OFDM systems , 1997, IEEE Trans. Signal Process..

[7]  K. Langer,et al.  513 Mbit/s Visible Light Communications Link Based on DMT-Modulation of a White LED , 2010, Journal of Lightwave Technology.

[8]  Harald Haas,et al.  Predistortion in Optical Wireless Transmission Using OFDM , 2009, 2009 Ninth International Conference on Hybrid Intelligent Systems.

[9]  Jun Rekimoto,et al.  ID CAM: a smart camera for scene capturing and ID recognition , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[10]  Ashwin Ashok,et al.  Spatially varying radiometric calibration for camera-display messaging , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[11]  Yoshifumi Kitamura,et al.  Interactive stereoscopic display for three or more users , 2001, SIGGRAPH.

[12]  Shoji Kawahito,et al.  A CMOS imager and 2-D light pulse receiver array for spatial optical communication , 2009, 2009 IEEE Asian Solid-State Circuits Conference.

[13]  Ramesh Raskar,et al.  Bokode: imperceptible visual tags for camera based interaction from a distance , 2009, ACM Trans. Graph..

[14]  Gang Zhang,et al.  Invisible barcode with optimized error correction , 2008, 2008 15th IEEE International Conference on Image Processing.

[15]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  Takeshi Naemura,et al.  Lumisight table: a face-to-face collaboration support system that optimizes direction of projected information to each stakeholder , 2004, CSCW.

[17]  Ramesh Raskar,et al.  VRCodes: Unobtrusive and active visual codes for interaction by exploiting rolling shutter , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[18]  Harald Haas,et al.  Using a CMOS camera sensor for visible light communication , 2012, 2012 IEEE Globecom Workshops.

[19]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[20]  Jane Yung-jen Hsu,et al.  Enabling beyond-surface interactions for interactive surface with an invisible projection , 2010, UIST.

[21]  Dina Katabi,et al.  PixNet: interference-free wireless links using LCD-camera pairs , 2010, MobiCom.

[22]  Oliver Bimber,et al.  Dynamic Adaptation of Projected Imperceptible Codes , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.