Image-Adaptive YOLO for Object Detection in Adverse Weather Conditions

Though deep learning-based object detection methods have achieved promising results on the conventional datasets, it is still challenging to locate objects from the low-quality images captured in adverse weather conditions. The existing methods either have difficulties in balancing the tasks of image enhancement and object detection, or often ignore the latent information beneficial for detection. To alleviate this problem, we propose a novel Image-Adaptive YOLO (IA-YOLO) framework, where each image can be adaptively enhanced for better detection performance. Specifically, a differentiable image processing (DIP) module is presented to take into account the adverse weather conditions for YOLO detector, whose parameters are predicted by a small convolutional neural network (CNN-PP). We learn CNN-PP and YOLOv3 jointly in an end-to-end fashion, which ensures that CNN-PP can learn an appropriate DIP to enhance the image for detection in a weakly supervised manner. Our proposed IA-YOLO approach can adaptively process images in both normal and adverse weather conditions. The experimental results are very encouraging, demonstrating the effectiveness of our proposed IAYOLO method in both foggy and low-light scenarios. The source code can be found at https://github.com/wenyyu/ImageAdaptive-YOLO.

[1]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Hao He,et al.  Exposure , 2017, ACM Trans. Graph..

[3]  Chee Seng Chan,et al.  Getting to Know Low-light Images with The Exclusively Dark Dataset , 2018, Comput. Vis. Image Underst..

[4]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Lei Xiang,et al.  Multi-Scale Boosted Dehazing Network With Dense Feature Fusion , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Xiaohui Yuan,et al.  An adaptive weak light image enhancement method , 2021, International Conference on Signal Processing Systems.

[8]  Lei Zhang,et al.  Learning Image-Adaptive 3D Lookup Tables for High Performance Photo Enhancement in Real-Time , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Shih-Chia Huang,et al.  DSNet: Joint Semantic Learning for Object Detection in Inclement Weather Conditions , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Zhongliang Jing,et al.  Domain Adaptive YOLO for One-Stage Cross-Domain Detection , 2021, ACML.

[11]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[12]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[13]  Deli Zhao,et al.  DeepExposure: Learning to Expose Photos with Asynchronously Reinforced Adversarial Learning , 2018, NeurIPS.

[14]  Dan Feng,et al.  Benchmarking Single-Image Dehazing and Beyond , 2017, IEEE Transactions on Image Processing.

[15]  Zeyun Yu,et al.  A fast and adaptive method for image contrast enhancement , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[16]  Sam Kwong,et al.  Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[18]  Luc Van Gool,et al.  Domain Adaptive Faster R-CNN for Object Detection in the Wild , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Katsushi Ikeuchi,et al.  Adherent Raindrop Modeling, Detectionand Removal in Video , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Qian Xu,et al.  A PSO and BFO-Based Learning Strategy Applied to Faster R-CNN for Object Detection in Autonomous Driving , 2019, IEEE Access.

[22]  A. Cantor Optics of the atmosphere--Scattering by molecules and particles , 1978, IEEE Journal of Quantum Electronics.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[25]  Jun Chen,et al.  GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[26]  Vishal M. Patel,et al.  Prior-Based Domain Adaptive Object Detection for Hazy and Rainy Conditions , 2019, ECCV.

[27]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[28]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[29]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[31]  Giovanni Ramponi,et al.  Image enhancement via adaptive unsharp masking , 2000, IEEE Trans. Image Process..

[32]  Jizheng Xu,et al.  AOD-Net: All-in-One Dehazing Network , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Luc Van Gool,et al.  The 2005 PASCAL Visual Object Classes Challenge , 2005, MLCW.

[34]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Shree K. Nayar,et al.  Vision and the Atmosphere , 2002, International Journal of Computer Vision.

[36]  Hayder Radha,et al.  Multiscale Domain Adaptive YOLO for Cross-Domain Object Detection , 2021, ICIP 2021.

[37]  Felix Heide,et al.  Hardware-in-the-Loop End-to-End Optimization of Camera Image Processing Pipelines , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.