Open PRAIRIE: Open Public Research Access Institutional Repository and Information Exchange

Illumination change is one of the challenges in image-based localization in smart cars application. To deal with illumination change, image conversion methods have been researched. However, these methods would lose the detail of objects in images. In this paper, we propose the Semantic Local Image Conversion (SLIC) model changing the appearance of local semantic objects in an image by categories at night. This enables the proposed model not to lose the detail of static objects in image conversion. As a result, it is expected that the proposed SLIC method has a better result in image-based localization. SLIC method uses static objects (i.e., traffic signs and street lamps) as categories for localization. The SLIC method is composed of two phases as (1) instance segmentation and (2) static objects conversion. Instance segmentation is utilized as a detector for static objects. In the conversion phase, the detected static objects are converted from the appearance of objects at night to objects at day. We then compare the visual inspection and the number of matching of converted objects with existed models (Pix2Pix with global pixels in the image and ToDayGAN). Overall, our model shows the better the result of image translation compared to Pix2Pix model and ToDayGAN models in both visual inspection and ORB matching cost.

[1]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[2]  Oisin Mac Aodha,et al.  Unsupervised Monocular Depth Estimation with Left-Right Consistency , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Wolfram Burgard,et al.  Robust visual SLAM across seasons , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Thomas Brox,et al.  Generating Images with Perceptual Similarity Metrics based on Deep Networks , 2016, NIPS.

[6]  Lars Hammarstrand,et al.  Long-Term Visual Localization Using Semantically Segmented Images , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Alexei A. Efros,et al.  Colorful Image Colorization , 2016, ECCV.

[8]  H. Jin Kim,et al.  Robust visual localization in changing lighting conditions , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Wojciech Zaremba,et al.  Improved Techniques for Training GANs , 2016, NIPS.

[10]  Hammam A. Alshazly,et al.  Image Features Detection, Description and Matching , 2016 .

[11]  Valérie Gouet-Brunet,et al.  A survey on Visual-Based Localization: On the benefit of heterogeneous data , 2018, Pattern Recognit..

[12]  Wolfram Burgard,et al.  Vision-based Markov localization across large perceptual changes , 2015, 2015 European Conference on Mobile Robots (ECMR).

[13]  Luc Van Gool,et al.  ComboGAN: Unrestrained Scalability for Image Domain Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[14]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[15]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[16]  Gordon Wyeth,et al.  SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights , 2012, 2012 IEEE International Conference on Robotics and Automation.

[17]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[18]  Paul Newman,et al.  Adversarial Training for Adverse Conditions: Robust Metric Localisation Using Appearance Transfer , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Peter I. Corke,et al.  Visual Place Recognition: A Survey , 2016, IEEE Transactions on Robotics.

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Shuicheng Yan,et al.  An HOG-LBP human detector with partial occlusion handling , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[22]  Paul Newman,et al.  From dusk till dawn: Localisation at night using artificial light sources , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Tie-Yan Liu,et al.  Dual Learning for Machine Translation , 2016, NIPS.

[24]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Rob Fergus,et al.  Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks , 2015, NIPS.

[27]  Emilio Garcia-Fidalgo,et al.  Vision-based topological mapping and localization methods: A survey , 2015, Robotics Auton. Syst..

[28]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[29]  Wei Zhang,et al.  Image Based Localization in Urban Environments , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[30]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Paul Newman,et al.  Illumination Invariant Imaging : Applications in Robust Vision-based Localisation , Mapping and Classification for Autonomous Vehicles , 2014 .

[32]  Luc Van Gool,et al.  Night-to-Day Image Translation for Retrieval-based Localization , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[33]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Swami Sankaranarayanan,et al.  Unsupervised Domain Adaptation for Semantic Segmentation with GANs , 2017, ArXiv.

[35]  Mohamed S. Shehata,et al.  Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images , 2017, ArXiv.

[36]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[37]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[38]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[39]  Masatoshi Okutomi,et al.  24/7 Place Recognition by View Synthesis , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Paul Newman,et al.  Detecting Loop Closure with Scene Sequences , 2007, International Journal of Computer Vision.