Predictive Driving in an Unstructured Scenario Using the Bundle Adjustment Algorithm

In this article, the autonomous driving problem in an unstructured scene is addressed using a model predictive control (MPC) scheme. The lack of scene structure makes the sensing problem challenging, in particular when considered within a control loop. To circumvent this difficulty, the bundle adjustment (BA) algorithm from computer vision is used to detect obstacles and compute a sparse representation of the environment. In one of the main results of this article, it is shown how this sparse representation can be cast as additional safety constraints to the MPC optimization. The MPC/BA combination is intuitively appealing since they both solve quadratic problems and also because the BA estimations trend to be more accurate when the resulting constraints become active in the MPC solver. This article contains a theoretical presentation of the control scheme and discusses implementation details. An example of the overall approach at work can be seen in https://youtu.be/aU46vpzDHso.

[1]  Anthony Tzes,et al.  Modelling and Control of Hyper-Redundant Micromanipulators for Obstacle Avoidance in an Unstructured Environment , 2015, J. Intell. Robotic Syst..

[2]  Ehud Rivlin,et al.  Monocular Sensing for Predictive Driving in an Unstructured Scenario , 2019, 2019 27th Mediterranean Conference on Control and Automation (MED).

[3]  Alberto Bemporad,et al.  Predictive Control for Linear and Hybrid Systems , 2017 .

[4]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[5]  Gheorghe Livint,et al.  Centralized model predictive control of autonomous driving vehicles with Lyapunov stability , 2016, 2016 20th International Conference on System Theory, Control and Computing (ICSTCC).

[6]  Ehud Rivlin,et al.  Finding the focus of expansion and estimating range using optical flow images and a matched filter , 2004, Machine Vision and Applications.

[7]  Ehud Rivlin,et al.  Estimating camera pose using Bundle Adjustment and Digital Terrain Model constraints , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Francesco Borrelli,et al.  A Learning-Based Framework for Velocity Control in Autonomous Driving , 2016, IEEE Transactions on Automation Science and Engineering.

[9]  Ehud Rivlin,et al.  Estimating Pose and Motion Using Bundle Adjustment and Digital Elevation Model Constraints , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[10]  Jonathan P. How,et al.  Real-Time Motion Planning With Applications to Autonomous Urban Driving , 2009, IEEE Transactions on Control Systems Technology.

[11]  Francesco Borrelli,et al.  Predictive Active Steering Control for Autonomous Vehicle Systems , 2007, IEEE Transactions on Control Systems Technology.

[12]  Stephen M. Erlien,et al.  Collision Avoidance and Stabilization for Autonomous Vehicles in Emergency Scenarios , 2017, IEEE Transactions on Control Systems Technology.

[13]  Alberto Bemporad,et al.  Spatial-Based Predictive Control and Geometric Corridor Planning for Adaptive Cruise Control Coupled With Obstacle Avoidance , 2018, IEEE Transactions on Control Systems Technology.

[14]  Khalid Saeed,et al.  Moving Object Detection Using Background Subtraction , 2014, SpringerBriefs in Computer Science.

[15]  Francesco Borrelli,et al.  Automated driving: The role of forecasts and uncertainty - A control perspective , 2015, Eur. J. Control.

[16]  Amnon Shashua,et al.  Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Saiful Islam,et al.  Mahalanobis Distance , 2009, Encyclopedia of Biometrics.

[18]  Stefano Di Cairano,et al.  The development of Model Predictive Control in automotive industry: A survey , 2012, 2012 IEEE International Conference on Control Applications.

[19]  Francesco Borrelli,et al.  MPC-Based Approach to Active Steering for Autonomous Vehicle Systems , 2005 .

[20]  E. Kim,et al.  Model predictive control strategy for smooth path tracking of autonomous vehicles with steering actuator dynamics , 2014 .

[21]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[22]  Yiqi Gao,et al.  Model Predictive Control for Autonomous and Semiautonomous Vehicles , 2014 .

[23]  Amnon Shashua,et al.  Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving , 2016, ArXiv.

[24]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[25]  Torsten Sattler,et al.  Obstacle detection for self-driving cars using only monocular cameras and wheel odometry , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Jianxiong Xiao,et al.  DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[27]  Kurt Konolige,et al.  Visual Odometry Using Sparse Bundle Adjustment on an Autonomous Outdoor Vehicle , 2006, AMS.

[28]  Javier Civera,et al.  1‐Point RANSAC for extended Kalman filtering: Application to real‐time structure from motion and visual odometry , 2010, J. Field Robotics.

[29]  Frank Dellaert,et al.  Incremental light bundle adjustment for robotics navigation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.