Monocular model-based 3D vehicle tracking for autonomous vehicles in unstructured environment

In this paper we describe a novel approach to model-based monocular vehicle tracking out of a moving vehicle using active vision. The designed algorithm can cope with cluttered color images, complex lighting conditions as well as partial occlusion of the leading vehicle and is able to detect and track a vehicle even within unstructured offroad environments. Thanks to the used 3D model which describes the characteristic vehicle geometry and appearance in terms of vertexes, edges and colored surfaces, no special visual markers are required. The knowledge of vehicle's geometry and appearance gained from the model are used within a particle filter to estimate the 6DoF position relative to the ego vehicle, thereby fusing edge as well as color information. We successfully use the proposed algorithm for pure vision based autonomous offroad convoy driving.

[1]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[2]  M. Himmelsbach,et al.  Real-time object classification in 3D point clouds using point feature histograms , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Stephen J. Maybank,et al.  Visual Surveillance for Moving Vehicles , 1998, International Journal of Computer Vision.

[4]  Massimo Bertozzi,et al.  Vehicle detection by means of stereo vision-based obstacles features extraction and monocular pattern analysis , 2006, IEEE Transactions on Image Processing.

[5]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Frank Dellaert,et al.  Robust car tracking using Kalman filtering and Bayesian templates , 1998, Other Conferences.

[7]  Patrick Rives,et al.  Vision-based Control for Car Platooning using Homography Decomposition , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[8]  Henrik I. Christensen,et al.  Real-time 3D model-based tracking using edge and keypoint features for robotic manipulation , 2010, 2010 IEEE International Conference on Robotics and Automation.

[9]  Sebastian Thrun,et al.  Model based vehicle detection and tracking for autonomous urban driving , 2009, Auton. Robots.

[10]  A. Muller,et al.  A model-based object following system , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[11]  Ronald Lumia,et al.  Vision-based robotic convoy driving , 1995, Machine Vision and Applications.

[12]  F. Bottiger,et al.  Truck platooning in mixed traffic , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[13]  L. Davis,et al.  Real-time multiple vehicle detection and tracking from a moving vehicle , 2000, Machine Vision and Applications.

[14]  Aurelio Piazzi,et al.  Visual perception of obstacles and vehicles for platooning , 2000, IEEE Trans. Intell. Transp. Syst..

[15]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[16]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  Paul E. Rybski,et al.  Obstacle Detection and Tracking for the Urban Challenge , 2009, IEEE Transactions on Intelligent Transportation Systems.

[18]  Alexander Barth,et al.  Estimating the Driving State of Oncoming Vehicles From a Moving Platform Using Stereo Vision , 2009, IEEE Transactions on Intelligent Transportation Systems.

[19]  Hans-Joachim Wünsche,et al.  Hybrid adaptive control of an active multi-focal vision system , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[20]  Luc Van Gool,et al.  Dynamic 3D Scene Analysis from a Moving Vehicle , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Edward Jones,et al.  Rear-Lamp Vehicle Detection and Tracking in Low-Exposure Color Video for Night Conditions , 2010, IEEE Transactions on Intelligent Transportation Systems.

[22]  Takeo Kanade,et al.  A robust shape model for multi-view car alignment , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Ernst D. Dickmanns,et al.  Dynamic Vision for Perception and Control of Motion , 2007 .

[24]  Hans-Hellmut Nagel,et al.  Initialization of Model-Based Vehicle Tracking in Video Sequences of Inner-City Intersections , 2007, International Journal of Computer Vision.

[25]  Zehang Sun,et al.  On-road vehicle detection: a review , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Li-Chen Fu,et al.  Vehicle Detection under Various Lighting Conditions by Incorporating Particle Filter , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[27]  Tieniu Tan,et al.  3-D model-based vehicle tracking , 2005, IEEE Transactions on Image Processing.

[28]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  Hans-Joachim Wünsche,et al.  A hybrid estimation approach for autonomous dirt road following using multiple clothoid segments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[30]  Michael Schmid.  An approach to model-based 3-D recognition of vehicles in real time by machine vision , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).