Some practical constraints and solutions for optical camera communication

Mobile wireless communication heavily relies on the radio frequency to convey message and data. However, its limited spectrum can hardly meet the demands for the future high data rate applications. Optical wireless communication, in particular visible light communication, opens up vast optical spectrum for communication, and meanwhile can retrofit the light sources as the communication transmitters in the existing working or living environments. In conjunction with the ubiquitous cameras in hand-held consumer electronics such as smartphones and pads, optical camera communication (OCC) further takes advantages of image sensors as the communication receivers and realizes low-cost communication systems. This article first provides an overview of OCC systems. It then addresses some practical constraints, ranging from sensor low frame rate and instability, rolling shutter readout, to visual qualities of displayed images and videos, and link blockage between the transmitter and receiver. Accordingly, it introduces existing and new solutions to deal with those constraints by data modulation, newly developed camera structures, post-processing of sensed signals and non-line of sight OCC as a new form. In particular, indirect paths by either the indoor surface reflection or the outdoor atmospheric scattering are explored for link connectivity under blockage. Finally, some future research directions are suggested. This article is part of the theme issue ‘Optical wireless communication’.

[1]  Hung-Yu Chen,et al.  Visible Light Communication Using Receivers of Camera Image Sensor and Solar Cell , 2016, IEEE Photonics Journal.

[2]  Mohsen Kavehrad,et al.  Human shadowing effect on indoor visible light communications channel characteristics , 2014 .

[3]  Marco Gruteser,et al.  Reading between the pixels: Photographic steganography for camera display messaging , 2016, 2017 IEEE International Conference on Computational Photography (ICCP).

[4]  Harald Haas,et al.  Spatial Modulation Applied to Optical Wireless Communications in Indoor LOS Environments , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[5]  Thomas D. C. Little,et al.  Performance of optical spatial modulation and spatial multiplexing with imaging receiver , 2014, 2014 IEEE Wireless Communications and Networking Conference (WCNC).

[6]  Fuqiang Liu,et al.  Vehicular Visible Light Communications with LED Taillight and Rolling Shutter Camera , 2014, 2014 IEEE 79th Vehicular Technology Conference (VTC Spring).

[7]  Hideki Aoyama,et al.  Line scan sampling for visible light communication: Theory and practice , 2015, 2015 IEEE International Conference on Communications (ICC).

[8]  Zhengyuan Xu,et al.  A Cloud-assisted 57.6 Kbps Image Sensor Communication System Using a Smartphone Camera , 2018, 2018 11th International Symposium on Communication Systems, Networks & Digital Signal Processing (CSNDSP).

[9]  Parth H. Pathak,et al.  ColorBars: increasing data rate of LED-to-camera communication using color shift keying , 2015, CoNEXT.

[10]  Chi-Wai Chow,et al.  Adaptive Thresholding Scheme for Demodulation of Rolling-Shutter Images Obtained in CMOS Image Sensor Based Visible Light Communications , 2018, IEEE Photonics Journal.

[11]  C. Chow,et al.  Enhancement of Signal Performance in LED Visible Light Communications Using Mobile Phone Camera , 2015, IEEE Photonics Journal.

[12]  J.M. Kahn,et al.  High-speed nondirective optical communication for wireless networks , 1991, IEEE Network.

[13]  Tianxing Li,et al.  Real-Time Screen-Camera Communication Behind Any Scene , 2015, MobiSys.

[14]  Chi-Wai Chow,et al.  Long distance non-line-of-sight (NLOS) visible light signal detection based on rolling-shutter-patterning of mobile-phone camera. , 2017, Optics express.

[15]  Yeong Min Jang,et al.  Survey on optical camera communications: challenges and opportunities , 2015 .

[16]  Kui Ren,et al.  Rain Bar: Robust Application-Driven Visual Communication Using Color Barcodes , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[17]  Dominic C. O'Brien,et al.  Visible light communications: Challenges and possibilities , 2008, 2008 IEEE 19th International Symposium on Personal, Indoor and Mobile Radio Communications.

[18]  Chi-Wai Chow,et al.  Color-Shift Keying and Code-Division Multiple-Access Transmission for RGB-LED Visible Light Communications Using Mobile Phone Camera , 2014, IEEE Photonics Journal.

[19]  C. Herrmann Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena , 2001, Experimental Brain Research.

[20]  Shoji Kawahito,et al.  LED and CMOS Image Sensor Based Optical Wireless Communication System for Automotive Applications , 2013, IEEE Photonics Journal.

[21]  Harald Haas,et al.  Optical Spatial Modulation , 2011, IEEE/OSA Journal of Optical Communications and Networking.

[22]  Gerald C. Dipiazza MILLIMETER WAVE COMMUNICATION SYSTEM , 2010 .

[23]  Liang Song,et al.  Multiscale Vascular Enhancement Filter Applied to In Vivo Morphologic and Functional Photoacoustic Imaging of Rat Ocular Vasculature , 2019, IEEE Photonics Journal.

[24]  Toshiaki Fujii,et al.  Overlay coding for road-to-vehicle visible light communication using LED array and high-speed camera , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[25]  Wei Huang,et al.  Design and experimental demonstration of a real-time 95kbps optical camera communication system , 2016, 2016 10th International Symposium on Communication Systems, Networks and Digital Signal Processing (CSNDSP).

[26]  Takaya Yamazato,et al.  Spatially-modulated space-time coding in visible light communications using 2×2 LED array , 2014, 2014 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).

[27]  Dina Katabi,et al.  PixNet: LCD-camera pairs as communication links , 2010, SIGCOMM '10.

[28]  Wissam Farjow,et al.  Shadowing effects on visible light communication channels , 2016, 2016 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[29]  Zabih Ghassemlooy,et al.  Undersampled Digital PAM Subcarrier Modulation for Optical Camera Communications , 2015 .

[30]  Trang Nguyen,et al.  Modulation and Coding Scheme (MCS) for Indoor Image Sensor Communication System , 2017, Wirel. Pers. Commun..

[31]  Chi-Wai Chow,et al.  Thresholding schemes for visible light communications with CMOS camera using entropy-based algorithms. , 2016, Optics express.

[32]  T. Fujii,et al.  Visible light road-to-vehicle communication using high-speed camera , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[33]  Dina Katabi,et al.  PixNet: interference-free wireless links using LCD-camera pairs , 2010, MobiCom.

[34]  Stefan Videv,et al.  Towards a 100 Gb / s visible light wireless access network , 2015 .

[35]  Cheng Peng,et al.  Vectorized Color Modulation for Covert Camera-Screen Communication , 2019, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[36]  Chi-Wai Chow,et al.  Single-input multiple-output visible light optical wireless communications supporting quality of service , 2015 .

[37]  Toshiaki Fujii,et al.  High-speed-camera image processing based LED traffic light detection for road-to-vehicle visible light communication , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[38]  Masao Nakagawa,et al.  Fundamental analysis for visible-light communication system using LED lights , 2004, IEEE Transactions on Consumer Electronics.

[39]  Zhengyuan Xu,et al.  The color shift key modulation with non-uniform signaling for visible light communication , 2012, 2012 1st IEEE International Conference on Communications in China Workshops (ICCC).

[40]  Takeshi Takaki,et al.  High-Frame-Rate Optical Flow System , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Tianxing Li,et al.  HiLight: Hiding Bits in Pixel Translucency Changes , 2015, MOCO.

[42]  Sang-Kook Han,et al.  Optical Pulse Width Modulated Multilevel Transmission in CIS-Based VLC , 2017, IEEE Photonics Technology Letters.

[43]  Zhaocheng Wang,et al.  Millimeter Wave Communication Systems: Huang/Millimeter Wave Communication System , 2011 .

[44]  Wei Huang,et al.  Design and implementation of a real-time CIM-MIMO optical camera communication system. , 2016, Optics express.

[45]  Harald Haas,et al.  Indoor optical wireless communication: potential and state-of-the-art , 2011, IEEE Communications Magazine.

[46]  Richard D. Roberts,et al.  Undersampled frequency shift ON-OFF keying (UFSOOK) for camera communications (CamCom) , 2013, 2013 22nd Wireless and Optical Communication Conference.

[47]  Anthony Rowe,et al.  Visual light landmarks for mobile devices , 2014, IPSN-14 Proceedings of the 13th International Symposium on Information Processing in Sensor Networks.

[48]  Nasir Saeed,et al.  Optical camera communications: Survey, use cases, challenges, and future trends , 2018, Phys. Commun..

[49]  Takaya Yamazato,et al.  Visible light communication systems using blue color difference modulation for digital signage , 2016, 2016 IEEE 27th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).

[50]  Yeong Min Jang,et al.  Invisible embedded techniques for optical camera communications , 2017, 2017 International Conference on Information Networking (ICOIN).

[51]  Zhi Chen,et al.  A survey on terahertz communications , 2019, China Communications.

[52]  Yoshihito Imai,et al.  Performance evaluation of high-speed visible light communication combining low-speed image sensor and polygon mirror in an outdoor environment , 2016, 2016 Eighth International Conference on Ubiquitous and Future Networks (ICUFN).

[53]  Xuan Tang,et al.  Experimental Demonstration of a 1024-QAM Optical Camera Communication System , 2016, IEEE Photonics Technology Letters.

[54]  Trang Nguyen,et al.  Region-of-Interest Signaling Vehicular System Using Optical Camera Communications , 2017, IEEE Photonics Journal.

[55]  Richard D. Roberts,et al.  A MIMO protocol for camera communications (CamCom) using undersampled frequency shift ON-OFF keying (UFSOOK) , 2013, 2013 IEEE Globecom Workshops (GC Wkshps).

[56]  Trang Nguyen,et al.  Current Status and Performance Analysis of Optical Camera Communication Technologies for 5G Networks , 2017, IEEE Access.

[57]  Yeong Min Jang,et al.  Trade-off Communication distance and Data rate of Rolling shutter OCC , 2019, 2019 Eleventh International Conference on Ubiquitous and Future Networks (ICUFN).

[58]  Brian M. Sadler,et al.  Ultraviolet Communications: Potential and State-Of-The-Art , 2008, IEEE Communications Magazine.

[59]  Yun Liu,et al.  Color-Shift Keying for Optical Camera Communication Using a Rolling Shutter Mode , 2019, IEEE Photonics Journal.

[60]  Murat Uysal,et al.  Survey on Free Space Optical Communication: A Communication Theory Perspective , 2014, IEEE Communications Surveys & Tutorials.

[61]  Mo Li,et al.  SoftLight: Adaptive visible light communication over screen-camera links , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[62]  Shoji Kawahito,et al.  A New Automotive VLC System Using Optical Communication Image Sensor , 2016, IEEE Photonics Journal.

[63]  Chunming Hu,et al.  Enhancing reliability to boost the throughput over screen-camera links , 2014, MobiCom.

[64]  Harald Haas,et al.  Indoor MIMO Optical Wireless Communication Using Spatial Modulation , 2010, 2010 IEEE International Conference on Communications.

[65]  Xuan Tang,et al.  Undersampled-PAM with subcarrier modulation for camera communications , 2015, 2015 Opto-Electronics and Communications Conference (OECC).

[66]  Chien-Hung Yeh,et al.  Secure Mobile-Phone Based Visible Light Communications With Different Noise-Ratio Light-Panel , 2018, IEEE Photonics Journal.

[67]  Xuan Tang,et al.  Undersampled phase shift ON-OFF keying for camera communication , 2014, 2014 Sixth International Conference on Wireless Communications and Signal Processing (WCSP).

[68]  Chi-Wai Chow,et al.  Visible light communication using mobile-phone camera with data rate higher than frame rate. , 2015, Optics express.

[69]  Chi-Wai Chow,et al.  Light Encryption Scheme Using Light-Emitting Diode and Camera Image Sensor , 2016, IEEE Photonics Journal.

[70]  Richard D. Roberts Space-time forward error correction for dimmable undersampled frequency shift ON-OFF keying camera communications (CamCom) , 2013, 2013 Fifth International Conference on Ubiquitous and Future Networks (ICUFN).

[71]  Weijie Liu,et al.  Predicted and Experimental Performance of a Long Distance Non-Line of Sight Image Sensor Communication System , 2018, 2018 IEEE International Conference on Communications Workshops (ICC Workshops).

[72]  Ashwin Ashok,et al.  Dynamic and invisible messaging for visual MIMO , 2012, 2012 IEEE Workshop on the Applications of Computer Vision (WACV).

[73]  Toshiaki Fujii,et al.  Experimental on Hierarchical Transmission Scheme for Visible Light Communication using LED Traffic Light and High-Speed Camera , 2007, 2007 IEEE 66th Vehicular Technology Conference.

[74]  Chang-Jun Ahn,et al.  Mobile Phone Camera-Based Indoor Visible Light Communications With Rotation Compensation , 2016, IEEE Photonics Journal.

[75]  Yeon-Ho Chung,et al.  Dual camera-based split shutter for high-rate and long-distance optical camera communications , 2016 .

[76]  Eric Monteiro,et al.  Constellation design for color-shift keying using interior point methods , 2012, 2012 IEEE Globecom Workshops.

[77]  Shoji Kawahito,et al.  Optical Vehicle-to-Vehicle Communication System Using LED Transmitter and Camera Receiver , 2014, IEEE Photonics Journal.

[78]  Thithanhnhan Le,et al.  Effect of blur Gaussian in MIMO optical camera communications , 2015, 2015 International Conference on Information and Communication Technology Convergence (ICTC).

[79]  Trang Nguyen,et al.  High-speed asynchronous Optical Camera Communication using LED and rolling shutter camera , 2015, 2015 Seventh International Conference on Ubiquitous and Future Networks.

[80]  Toshiaki Fujii,et al.  Erasure coding for road-to-vehicle visible light communication systems , 2011, 2011 IEEE Consumer Communications and Networking Conference (CCNC).

[81]  Steve Hranilovic,et al.  Design and Implementation of Color-Shift Keying for Visible Light Communications , 2014, Journal of Lightwave Technology.

[82]  Mohamed-Slim Alouini,et al.  FSO-Based Vertical Backhaul/Fronthaul Framework for 5G+ Wireless Networks , 2016, IEEE Communications Magazine.

[83]  Cong Wang,et al.  SBVLC: Secure barcode-based visible light communication for smartphones , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[84]  Yuncai Wang,et al.  Enhancing the Brightness of Quantum Dot Light-Emitting Diodes by Multilayer Heterostructures , 2016, IEEE Photonics Journal.

[85]  Zabih Ghassemlooy,et al.  Undersampled-Based Modulation Schemes for Optical Camera Communications , 2018, IEEE Communications Magazine.

[86]  Ki-Doo Kim,et al.  Time-Sharing-Based Synchronization and Performance Evaluation of Color-Independent Visual-MIMO Communication , 2018, Sensors.

[87]  Guoliang Xing,et al.  COBRA: color barcode streaming for smartphone systems , 2012, MobiSys '12.

[88]  Lih Chieh Png,et al.  Experimental Demonstration of RGB LED-Based Optical Camera Communications , 2015, IEEE Photonics Journal.

[89]  Zhilin Xu,et al.  Microfiber-Based Inline Mach–Zehnder Interferometer for Dual-Parameter Measurement , 2015, IEEE Photonics Journal.

[90]  Xiangjun Xin,et al.  Performance Comparison of PS Star-16QAM and PS Square-Shaped 16QAM (Square-16QAM) , 2017, IEEE Photonics Journal.

[91]  Hai-Han Lu,et al.  A 56 Gb/s PAM4 VCSEL-Based LiFi Transmission With Two-Stage Injection-Locked Technique , 2017, IEEE Photonics Journal.

[92]  Yen-Ting Chen,et al.  Beacon Jointed Packet Reconstruction Scheme for Mobile-Phone Based Visible Light Communications Using Rolling Shutter , 2017, IEEE Photonics Journal.

[93]  Wenjun Hu,et al.  LightSync: unsynchronized visual communication over screen-camera links , 2013, MobiCom.

[94]  Ashwin Ashok,et al.  High-rate flicker-free screen-camera communication with spatially adaptive embedding , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[95]  Jie Wang,et al.  Demonstration of a covert camera-screen communication system , 2017, 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC).

[96]  Tao Shang,et al.  Study on VLC Channel Modeling Under Random Shadowing , 2017, IEEE Photonics Journal.

[97]  Jun Luo,et al.  CeilingCast: Energy efficient and location-bound broadcast through LED-camera communication , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[98]  Kang-Il Ahn,et al.  Color Intensity Modulation for Multicolored Visible Light Communications , 2012, IEEE Photonics Technology Letters.

[99]  Toshiaki Fujii,et al.  Tracking an LED array transmitter for visible light communications in the driving situation , 2010, 2010 7th International Symposium on Wireless Communication Systems.

[100]  S. Hranilovic,et al.  A pixelated MIMO wireless optical communication system , 2006, IEEE Journal of Selected Topics in Quantum Electronics.

[101]  Takaya Yamazato,et al.  Layered Transmission of Space-Time Coded Signals for Image-Sensor-Based Visible Light Communications , 2015, Journal of Lightwave Technology.

[102]  Ki-Doo Kim,et al.  Synchronization method for color-independent visual-MIMO communication , 2017, 2017 IEEE International Conference on Consumer Electronics (ICCE).

[103]  Rüdiger Kays,et al.  Modulation concepts for high-rate display-camera data transmission , 2017, 2017 IEEE International Conference on Communications (ICC).

[104]  V.W.S. Chan,et al.  Free-Space Optical Communications , 2006, Journal of Lightwave Technology.

[105]  Zabih Ghassemlooy,et al.  Non-line-of-sight 2 × N indoor optical camera communications. , 2018, Applied optics.

[106]  Trang Nguyen,et al.  Asynchronous Scheme for Optical Camera Communication-Based Infrastructure-to-Vehicle Communication , 2015, Int. J. Distributed Sens. Networks.

[107]  Mauro Biagi,et al.  Non-Line-of-Sight MIMO Space-Time Division Multiplexing Visible Light Optical Camera Communications , 2019, Journal of Lightwave Technology.

[108]  Wei Huang,et al.  Characteristics and Performance of Image Sensor Communication , 2017, IEEE Photonics Journal.

[109]  Nam-Tuan Le,et al.  MIMO Architecture for Optical Camera Communications , 2017 .

[110]  Insoo Koo,et al.  Sensor Clustering and Sensing Technology for Optimal Throughput of Sensor-Aided Cognitive Radio Networks Supporting Multiple Licensed Channels , 2015, Int. J. Distributed Sens. Networks.

[111]  Takaya Yamazato,et al.  Technical Issues on IEEE 802.15.7m Image Sensor Communication Standardization , 2018, IEEE Communications Magazine.

[112]  Guobin Shen,et al.  InFrame++: Achieve Simultaneous Screen-Human Viewing and Hidden Screen-Camera Communication , 2015, MobiSys.

[113]  Toshiaki Fujii,et al.  Improved Decoding Methods of Visible Light Communication System for ITS Using LED Array and High-Speed Camera , 2010, 2010 IEEE 71st Vehicular Technology Conference.

[114]  Guobin Shen,et al.  InFrame: Multiflexing Full-Frame Visible Communication Channel for Humans and Devices , 2014, HotNets.

[115]  Anna Maria Vegni,et al.  Interference cancellation in MIMO NLOS optical-camera-communication-based intelligent transport systems. , 2019, Applied optics.

[116]  Harald Haas,et al.  Using a CMOS camera sensor for visible light communication , 2012, 2012 IEEE Globecom Workshops.

[117]  Takehiro Moriya,et al.  Flexible Synchronization in Optical Camera Communication with On-Off Keying , 2017, 2017 IEEE Globecom Workshops (GC Wkshps).

[118]  Harald Haas,et al.  Visible light communication using OFDM , 2006, 2nd International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities, 2006. TRIDENTCOM 2006..

[119]  P. Mégret,et al.  Multicarrier approaches for high-baudrate optical-fiber transmission systems with a single coherent receiver , 2017, 2017 IEEE Photonics Conference (IPC).

[120]  Dominic C. O'Brien,et al.  Wireless Myths, Realities, and Futures: From 3G/4G to Optical and Quantum Wireless , 2012, Proceedings of the IEEE.

[121]  Jian Song,et al.  A VLC-based robust ID broadcasting indoor location system for mobile devices , 2016, 2016 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[122]  Qi Wang,et al.  Visible Light Communications : Modulation and Signal Processing , 2017 .

[123]  Jong Tae Kim,et al.  Smartphone Image Receiver Architecture for Optical Camera Communication , 2017, Wirel. Pers. Commun..

[124]  Takaya Yamazato Visible Light Communications Using LED Array and High-speed Camera for ITS Applications , 2009 .