Making of Night Vision: Object Detection Under Low-Illumination

Object detection has so far achieved great success. However, almost all of current state-of-the-art methods focus on images with normal illumination, while object detection under low-illumination is often ignored. In this paper, we have extensively investigated several important issues related to the challenge low-illumination detection task, such as the importance of illumination on detection, the applicabilities of illumination enhancement on low-illumination object detection task, and the influences of illumination balanced dataset and model’s parameters initialization, etc. We further have proposed a Night Vision Detector (NVD) with specifically designed feature pyramid network and context fusion network for object detection under low-illuminance. Through conducting comprehensive experiments on a public real low-illuminance scene dataset ExDARK and a selected normal-illumination counterpart COCO*, we on one hand have reached some valuable conclusions for reference, on the other hand, have found specific solutions for low-illumination object detection. Our strategy improves detection performance by 0.5%~2.8% higher than basic model on all standard COCO evaluation criterions. Our work can be taken as effective baseline and shed light to future studies on low-illumination detection.

[1]  Zia-ur Rahman,et al.  Multi-scale retinex for color image enhancement , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[2]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[3]  Xiaojie Guo,et al.  LIME: A Method for Low-light IMage Enhancement , 2016, ACM Multimedia.

[4]  Fatih Murat Porikli,et al.  LightenNet: A Convolutional Neural Network for weakly illuminated image enhancement , 2018, Pattern Recognit. Lett..

[5]  Calle Lejdfors,et al.  Adaptive enhancement and noise reduction in very low light-level video , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Rogério Schmidt Feris,et al.  A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection , 2016, ECCV.

[7]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[8]  Chee Seng Chan,et al.  Getting to Know Low-light Images with The Exclusively Dark Dataset , 2018, Comput. Vis. Image Underst..

[9]  David R. Bull,et al.  Automatic contrast enhancement of low-light images based on local statistics of wavelet coefficients , 2010, 2010 IEEE International Conference on Image Processing.

[10]  Wei Liu,et al.  DSSD : Deconvolutional Single Shot Detector , 2017, ArXiv.

[11]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[12]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[13]  Naoto Yokoya,et al.  An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing , 2018, IEEE Transactions on Image Processing.

[14]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[17]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[18]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[20]  Xiaoyan Sun,et al.  Structure-Revealing Low-Light Image Enhancement Via Robust Retinex Model , 2018, IEEE Transactions on Image Processing.

[21]  Jia Xu,et al.  Learning to See in the Dark , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Yunhong Wang,et al.  Receptive Field Block Net for Accurate and Fast Object Detection , 2017, ECCV.

[23]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[25]  Tao Lu,et al.  Low-light image enhancement using CNN and bright channel prior , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[26]  Ding Liu,et al.  EnlightenGAN: Deep Light Enhancement Without Paired Supervision , 2019, IEEE Transactions on Image Processing.

[27]  Jie Ma,et al.  MSR-net: Low-light Image Enhancement Using Deep Convolutional Network , 2017, ArXiv.

[28]  Jiangtao Wen,et al.  Fast efficient algorithm for enhancement of low lighting video , 2010, 2011 IEEE International Conference on Multimedia and Expo.

[29]  Yu Zhang,et al.  Self-supervised Image Enhancement Network: Training with Low Light Images Only , 2020, ArXiv.

[30]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[31]  Jocelyn Chanussot,et al.  Fourier-Based Rotation-Invariant Feature Boosting: An Efficient Framework for Geospatial Object Detection , 2019, IEEE Geoscience and Remote Sensing Letters.

[32]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[33]  Jocelyn Chanussot,et al.  ORSIm Detector: A Novel Object Detection Framework in Optical Remote Sensing Imagery Using Spatial-Frequency Channel Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Chen Wei,et al.  Deep Retinex Decomposition for Low-Light Enhancement , 2018, BMVC.

[35]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).