Stereo vision-based vehicle localization in point cloud maps using multiswarm particle swarm optimization

We propose a vision-based localization algorithm with multiswarm particle swarm optimization for driving an autonomous vehicle. With stereo vision, the vehicle can be localized within a 3D point cloud map using the particle swarm optimization. For vehicle localization, the GPS (global positioning system)-based algorithms are often affected by the certain conditions resulting in intermittent missing signal. We address this issue in vehicle localization by using stereo vision in addition to the GPS information. The depth-based localization is formulated as an optimization-based tracking problem. Virtual depth images generated from the point cloud are matched with the online stereo depth images using the particle swarm optimization. The virtual depth images are generated from the point cloud using a series of coordinate transforms. We propose a novel computationally efficient tracker, i.e., a multiswarm particle swarm optimization-based algorithm. The tracker is initialized with GPS information and employs a Kalman filter in the bootstrapping phase. The Kalman filter stabilizes the GPS information in this phase and, subsequently, initializes the online tracker. The proposed localization algorithm is validated with acquired datasets from driving tests. A detailed comparative and parametric analysis is conducted in the experiments. The experimental results demonstrate the effectiveness and robustness of the proposed algorithm for vehicle localization, which advances the state of the art for autonomous driving.

[1]  Vijay John,et al.  Markerless human articulated tracking using hierarchical particle swarm optimisation , 2010, Image Vis. Comput..

[2]  Joaquim Salvi,et al.  The SLAM problem: a survey , 2008, CCIA.

[3]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Lahcen Koutti,et al.  Fast edge-based stereo matching approach for road applications , 2017, Signal Image Video Process..

[5]  Y. Yang,et al.  GPS/INS Data Fusion for Land Vehicle Localization , 2006, The Proceedings of the Multiconference on "Computational Engineering in Systems Applications".

[6]  Frank Dellaert,et al.  Map-based priors for localization , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[7]  David Filliat,et al.  Qualitative Localization using Vision and Odometry for Path Following in Topo-metric Maps , 2011, ECMR.

[8]  Zheng Liu,et al.  Automatic calibration and registration of lidar and stereo camera without calibration objects , 2015, 2015 IEEE International Conference on Vehicular Electronics and Safety (ICVES).

[9]  Mohinder S. Grewal,et al.  Global Positioning Systems, Inertial Navigation, and Integration , 2000 .

[10]  Zheng Liu,et al.  Registration of GPS and Stereo Vision for Point Cloud Localization in Intelligent Vehicles Using Particle Swarm Optimization , 2017, ICSI.

[11]  Matthew N. Dailey,et al.  Simultaneous Localization and Mapping with Stereo Vision , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[12]  Gerd Wanielik,et al.  High-accurate vehicle localization using digital maps and coherency images , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[13]  Zheng Liu,et al.  Real-time road surface and semantic lane estimation using deep features , 2018, Signal Image Video Process..

[14]  Zheng Liu,et al.  Fusion of thermal and visible cameras for the application of pedestrian detection , 2017, Signal Image Video Process..

[15]  Ming Zhu,et al.  Obstacle detection in single images with deep neural networks , 2016, Signal Image Video Process..

[16]  Daniel Maier,et al.  Real-time navigation in 3D environments based on depth camera data , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[17]  Seiichi Mita,et al.  Real-time Dense Disparity Estimation based on Multi-Path Viterbi for Intelligent Vehicle Applications , 2014, BMVC.

[18]  Giovanni Ulivi,et al.  An outdoor navigation system using GPS and inertial platform , 2002 .

[19]  Oscar L. Colombo,et al.  Ephemeris errors of GPS satellites , 1986 .

[20]  Seiichi Mita,et al.  Lidar scan feature for localization with highly precise 3-D map , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[21]  Luisa Franconi,et al.  Comparison of a genetic algorithm and simulated annealing in an application to statistical image reconstruction , 1997, Stat. Comput..

[22]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[23]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Akihiro Takeuchi,et al.  On-Road Multivehicle Tracking Using Deformable Object Model and Particle Filter With Improved Likelihood Estimation , 2012, IEEE Transactions on Intelligent Transportation Systems.