On the Impact of Varying Region Proposal Strategies for Raindrop Detection and Classification Using Convolutional Neural Networks

The presence of raindrop induced image distortion has a significant negative impact on the performance of a wide range of all-weather visual sensing applications including within the increasingly important contexts of visual surveillance and vehicle autonomy. A key part of this problem is robust raindrop detection such that the potential for performance degradation in effected image regions can be identified. Here we address the problem of raindrop detection in colour video imagery by considering three varying region proposal approaches with secondary classification via a number of novel convolutional neural network architecture variants. This is verified over an extensive dataset with in-frame raindrop annotation to achieve maximal 0.95 detection accuracy with minimal false positives compared to prior work. Our approach is evaluated under a range of environmental conditions typical of all-weather automotive visual sensing applications.

[1]  Toby P. Breckon,et al.  Quantitative Evaluation of Stereo Visual Odometry for Autonomous Vessel Localisation in Inland Waterway Sensing Applications , 2015, Sensors.

[2]  Martin Roser,et al.  Raindrop detection on car windshields using geometric-photometric environment construction and intensity-based correlation , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[3]  Toby P. Breckon,et al.  Multi-modal target detection for autonomous wide area search and surveillance , 2013, Optics/Photonics in Security and Defence.

[4]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[7]  Anton Kummert,et al.  Vision-based rain sensing with an in-vehicle camera , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[8]  Ioannis Katramados,et al.  Dense gradient-based features (DEGRAF) for computationally efficient and invariant feature extraction in real-time applications , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[9]  Toby P. Breckon,et al.  An evaluation of region based object detection strategies within X-ray baggage security imagery , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[10]  Qi Wu,et al.  Raindrop detection and removal using salient visual features , 2012, 2012 19th IEEE International Conference on Image Processing.

[11]  M.L. Eichner,et al.  Integrated speed limit detection and recognition from real-time video , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[12]  Toby P. Breckon,et al.  Improved raindrop detection using combined shape and saliency descriptors with scene context isolation , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[13]  Andreas Geiger,et al.  Video-based raindrop detection for improved image registration , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[14]  Atsushi Yamashita,et al.  A virtual wiper - restoration of deteriorated images by using multiple cameras , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[15]  Toby P. Breckon,et al.  DepthComp: Real-time Depth Image Completion Based on Prior Semantic Scene Segmentation , 2017, BMVC.

[16]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Katsushi Ikeuchi,et al.  Adherent Raindrop Detection and Removal in Video , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Toby P. Breckon,et al.  From On-Road to Off: Transfer Learning Within a Deep Convolutional Neural Network for Segmentation and Classification of Off-Road Scenes , 2016, ECCV Workshops.

[19]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[21]  Toby P. Breckon,et al.  A foreground object based quantitative assessment of dense stereo approaches for use in automotive environments , 2013, 2013 IEEE International Conference on Image Processing.

[22]  Atsushi Yamashita,et al.  Removal of adherent noises from images of dynamic scenes by using a pan-tilt camera , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[23]  Toby P. Breckon,et al.  Generalized dynamic object removal for dense stereo vision based scene mapping using synthesised optical flow , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[24]  Luc Van Gool,et al.  SEEDS: Superpixels Extracted via Energy-Driven Sampling , 2012, ECCV.

[25]  Shree K. Nayar,et al.  Detection and removal of rain from videos , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[26]  Toby P. Breckon,et al.  Real-time construction and visualisation of drift-free video mosaics from unconstrained camera motion , 2015 .

[27]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[28]  I. Ide,et al.  Rainy weather recognition from in-vehicle camera images for driver assistance , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[29]  Atsushi Yamashita,et al.  Removal of adherent noises from image sequences by spatio-temporal image processing , 2008, 2008 IEEE International Conference on Robotics and Automation.