Real-time multiple vehicle detection and tracking from a moving vehicle

Abstract. A real-time vision system has been developed that analyzes color videos taken from a forward-looking video camera in a car driving on a highway. The system uses a combination of color, edge, and motion information to recognize and track the road boundaries, lane markings and other vehicles on the road. Cars are recognized by matching templates that are cropped from the input data online and by detecting highway scene features and evaluating how they relate to each other. Cars are also detected by temporal differencing and by tracking motion parameters that are typical for cars. The system recognizes and tracks road boundaries and lane markings using a recursive least-squares filter. Experimental results demonstrate robust, real-time car detection and tracking over thousands of image frames. The data includes video taken under difficult visibility conditions.

[1]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[2]  Takeo Kanade,et al.  Vision and Navigation for the Carnegie-Mellon Navlab , 1987 .

[3]  Martial Hebert,et al.  Vision and navigation for the Carnegie-Mellon Navlab , 1988 .

[4]  Charles E. Thorpe,et al.  Vision and Navigation , 1990 .

[5]  Ernst D. Dickmanns,et al.  Recursive 3-D Road and Relative Ego-State Recognition , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  I. Masaki,et al.  Vision-based vehicle guidance , 1992, Proceedings of the 1992 International Conference on Industrial Electronics, Control, Instrumentation, and Automation.

[7]  C..  Vision-based Vehicle Guidance , 1992, Springer Series in Perception Engineering.

[8]  W. Seelen,et al.  Intensity and edge-based symmetry detection with an application to car-following , 1993 .

[9]  Ashok K. Agrawala,et al.  Design and implementation of Maruti-II , 1993 .

[10]  Charles E. Thorpe,et al.  SCARF: a color vision system that tracks roads and intersections , 1993, IEEE Trans. Robotics Autom..

[11]  Dirk Dickmanns,et al.  Multiple object recognition and scene interpretation for autonomous road vehicle guidance , 1994, Proceedings of the Intelligent Vehicles '94 Symposium.

[12]  Volker Graefe,et al.  Visual recognition of obstacles on roads , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[13]  Reinhold Behringer,et al.  Simultaneous estimation of pitch angle and lane width from the video image of a marked road , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[14]  Jitendra Malik,et al.  Robust Multiple Car Tracking with Occlusion Reasoning , 1994, ECCV.

[15]  Stephen M. Smith,et al.  ASSET-2: Real-Time Motion Segmentation and Shape Tracking , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  K. Sasakawa,et al.  The detection of lane boundary markings using the modified spoke filter , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[17]  Jitendra Malik,et al.  An integrated stereo-based approach to automatic vehicle guidance , 1995, Proceedings of IEEE International Conference on Computer Vision.

[18]  W. Kruger,et al.  Real-time estimation and tracking of optical flow vectors for obstacle detection , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[19]  T. Suzuki,et al.  A real-time vision for intelligent vehicles , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[20]  H.-H. Nagel,et al.  Model-based recognition of intersections and lane structures , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[21]  Dean A. Pomerleau,et al.  RALPH: rapidly adapting lateral position handler , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[22]  F. Bottiger,et al.  Truck platooning in mixed traffic , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[23]  Alberto Broggi.  Parallel and local feature extraction: a real-time approach to road boundary detection , 1995, IEEE Trans. Image Process..

[24]  Thomas Zielke,et al.  An integrated obstacle detection framework for intelligent cruise control on motorways , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[25]  Karl C. Kluge,et al.  Statistical characterization of the visual characteristics of painted lane markings , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[26]  W. J. Gillner,et al.  Motion based vehicle detection on motorways , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[27]  Sridhar Lakshmanan,et al.  A deformable-template approach to lane detection , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[28]  I. Masaki,et al.  A sub-pixel stereo vision system for cost-effective intelligent vehicle applications , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[29]  Mahmood Fathy,et al.  A Window-based Edge Detection Technique for Measuring Road Traffic Parameters in Real-Time , 1995, Real Time Imaging.

[30]  Margrit Betke,et al.  Fast object recognition in noisy images using simulated annealing , 1995, Proceedings of IEEE International Conference on Computer Vision.

[31]  Andrea Giachetti,et al.  Dynamic segmentation of traffic scenes , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[32]  W. von Seelen,et al.  Real-time vehicle tracking and classification , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[33]  Hans A. Hansson,et al.  BASEMENT: A distributed real-time architecture for vehicle applications , 1995, Proceedings Real-Time Technology and Applications Symposium.

[34]  Larry S. Davis,et al.  Multiple vehicle detection and tracking in hard real-time , 1996, Proceedings of Conference on Intelligent Vehicles.

[35]  Volker Graefe,et al.  A novel approach for the detection of vehicles on freeways by real-time vision , 1996, Proceedings of Conference on Intelligent Vehicles.

[36]  Markus Maurer,et al.  A compact vision system for road vehicle guidance , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[37]  Thierry Pun,et al.  Combining Multiple Motion Estimates for Vehicle Tracking , 1996, ECCV.

[38]  Masanori Mizoguchi,et al.  Traffic measurement with a roadside vision system-individual tracking of overlapped vehicles , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[39]  Jitendra Malik,et al.  Tracking vehicles in congested traffic , 1996, Proceedings of Conference on Intelligent Vehicles.

[40]  Larry S. Davis,et al.  Highway scene analysis in hard real-time , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[41]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[42]  Wen-Liang Chen,et al.  Estimations of previewed road curvatures and vehicular motion by a vision-based data fusion scheme , 1997, Machine Vision and Applications.

[43]  Thomas Ertl,et al.  Computer Graphics - Principles and Practice, 3rd Edition , 2014 .

[44]  Dean A. Pomerleau,et al.  Visibility estimation from a moving vehicle using the RALPH vision system , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[45]  W. Enkelmann,et al.  Robust obstacle detection and tracking by motion analysis , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[46]  Jitendra Malik,et al.  A real-time computer vision system for measuring traffic parameters , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Keith Redmill,et al.  A simple vision system for lane keeping , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[48]  Frank Dellaert,et al.  Model-based car tracking integrated with a road-follower , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[49]  Frank Dellaert,et al.  Robust car tracking using Kalman filtering and Bayesian templates , 1998, Other Conferences.

[50]  Margrit Betke,et al.  HIGHWAY SCENE ANALYSIS FROM A MOVING VEHICLE UNDER REDUCED VISIBILITY CONDITIONS , 1998 .

[51]  F. Collange,et al.  3D LOCALIZATION OF A CAR OBSERVED THROUGH A MONOCULAR VIDEO CAMERA , 1998 .

[52]  Roldano Cattoni,et al.  Visual routines for real-time monitoring of vehicle behavior , 1998, Machine Vision and Applications.

[53]  Thomas Kalinke,et al.  A Texture-based Object Detection and an adaptive Model-based Classification , 1998 .

[54]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[55]  Margrit Betke,et al.  Information-conserving object recognition , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[56]  W. Krüger.  Robust real-time ground plane motion compensation from a moving vehicle , 1999, Machine Vision and Applications.

[57]  Dean A. Pomerleau,et al.  Overtaking vehicle detection using implicit optical flow , 1999 .

[58]  Gianni Conte,et al.  Automatic Vehicle Guidance: the Experience of the ARGO Autonomous Vehicle , 1999 .

[59]  Charles E. Thorpe,et al.  A trinocular stereo system for highway obstacle detection , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[60]  J. D. Crisman,et al.  Vehicle detection in color images , 1999 .