This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS 1 Integrated Lane and Vehicle Detection, Localization,

In this paper, we introduce a synergistic approach to integrated lane and vehicle tracking for driver assistance. The approach presented in this paper results in a final system that improves on the performance of both lane tracking and vehicle tracking modules. Further, the presented approach introduces a novel approach to localizing and tracking other vehicles on the road with respect to lane position, which provides information on higher contextual relevance that neither the lane tracker nor vehicle tracker can provide by itself. Improvements in lane tracking and vehicle tracking have been extensively quantified. Integrated system performance has been validated on real-world highway data. Without specific hardware and software optimizations, the fully implemented system runs at near-real-time speeds of 11 frames per second.

[1]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[2]  Mohan M. Trivedi,et al.  Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation , 2006, IEEE Transactions on Intelligent Transportation Systems.

[3]  Mohan M. Trivedi,et al.  On the Roles of Eye Gaze and Head Dynamics in Predicting Driver's Intent to Change Lanes , 2009, IEEE Transactions on Intelligent Transportation Systems.

[4]  Pierre Charbonnier,et al.  Evaluation of Road Marking Feature Extraction , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[5]  Mohan M. Trivedi,et al.  Learning multi-lane trajectories using vehicle-based vision , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[6]  Paulo Peixoto,et al.  On Exploration of Classifier Ensemble Synergism in Pedestrian Detection , 2010, IEEE Transactions on Intelligent Transportation Systems.

[7]  Mohan M. Trivedi,et al.  On Color-, Infrared-, and Multimodal-Stereo Approaches to Pedestrian Detection , 2007, IEEE Transactions on Intelligent Transportation Systems.

[8]  A. Lopez,et al.  3D vehicle sensor based on monocular vision , 2005, Proceedings. 2005 IEEE Intelligent Transportation Systems, 2005..

[9]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[10]  Akihiro Takeuchi,et al.  On-road vehicle tracking using deformable object model and particle filter with integrated likelihoods , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[11]  Pei-Yung Hsiao,et al.  Tracking and detection of lane and vehicle integrating lane and vehicle information using PDAF tracking model , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[12]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[13]  Li-Chen Fu,et al.  Vehicle Detection under Various Lighting Conditions by Incorporating Particle Filter , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[14]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision , 2004 .

[15]  Mirko Meuter,et al.  A novel approach to lane detection and tracking , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[16]  A. Kummert,et al.  A vehicle detection system based on Haar and Triangle features , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[17]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[18]  Margaret M. Peden,et al.  World Report on Road Traffic Injury Prevention , 2004 .

[19]  Mohan M. Trivedi,et al.  A Novel Active Heads-Up Display for Driver Assistance , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Ho Gi Jung,et al.  A New Approach to Urban Pedestrian Detection for Automatic Braking , 2009, IEEE Transactions on Intelligent Transportation Systems.

[21]  Mohan M. Trivedi,et al.  Turn-Intent Analysis Using Body Pose for Intelligent Driver Assistance , 2006, IEEE Pervasive Computing.

[22]  Christoph Stiller,et al.  Kalman Particle Filter for lane recognition on rural roads , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[23]  Mohan M. Trivedi,et al.  Improved vision-based lane tracker performance using vehicle localization , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[24]  Joan Serrat,et al.  Robust Lane Lines Detection and Quantitative Assessment , 2007, IbPRIA.

[25]  Sergiu Nedevschi,et al.  Probabilistic Lane Tracking in Difficult Road Scenarios Using Stereovision , 2009, IEEE Transactions on Intelligent Transportation Systems.

[26]  A. Jazayeri,et al.  Vehicle Detection and Tracking in Car Video Based on Motion Model , 2011, IEEE Transactions on Intelligent Transportation Systems.

[27]  Anton Kummert,et al.  An evolutionary optimized vehicle tracker in collaboration with a detection system , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[28]  Mohan M. Trivedi,et al.  A General Active-Learning Framework for On-Road Vehicle Recognition and Tracking , 2010, IEEE Transactions on Intelligent Transportation Systems.

[29]  Zehang Sun,et al.  Monocular precrash vehicle detection: features and classifiers , 2006, IEEE Transactions on Image Processing.

[30]  Mohan M. Trivedi,et al.  Lane Tracking with Omnidirectional Cameras: Algorithms and Evaluation , 2007, EURASIP J. Embed. Syst..

[31]  Dariu Gavrila,et al.  A mixed generative-discriminative framework for pedestrian classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Tarak Gandhi,et al.  Looking-In and Looking-Out of a Vehicle: Computer-Vision-Based Enhanced Vehicle Safety , 2007, IEEE Transactions on Intelligent Transportation Systems.

[33]  Mohan M. Trivedi,et al.  Active learning for on-road vehicle detection: a comparative study , 2014, Machine Vision and Applications.

[34]  Mohan M. Trivedi,et al.  Lane Change Intent Analysis Using Robust Operators and Sparse Bayesian Learning , 2005, IEEE Transactions on Intelligent Transportation Systems.

[35]  Mohan M. Trivedi,et al.  Combining monocular and stereo-vision for real-time vehicle ranging and tracking on multilane highways , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[36]  Larry S. Davis,et al.  Vehicle Detection Using Partial Least Squares , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Mohan M. Trivedi,et al.  Real-time vehicle detection using parts at intersections , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[38]  Edward Jones,et al.  Rear-Lamp Vehicle Detection and Tracking in Low-Exposure Color Video for Night Conditions , 2010, IEEE Transactions on Intelligent Transportation Systems.

[39]  Tarak Gandhi,et al.  Parametric ego-motion estimation for vehicle surround analysis using an omnidirectional camera , 2005, Machine Vision and Applications.

[40]  Mohan M. Trivedi,et al.  Unsupervised learning of motion patterns of rear surrounding vehicles , 2009, 2009 IEEE International Conference on Vehicular Electronics and Safety (ICVES).

[41]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[42]  Mohan M. Trivedi,et al.  Driver Behavior and Situation Aware Brake Assistance for Intelligent Vehicles , 2007, Proceedings of the IEEE.

[43]  Saturnino Maldonado-Bascón,et al.  Goal Evaluation of Segmentation Algorithms for Traffic Sign Recognition , 2010, IEEE Transactions on Intelligent Transportation Systems.

[44]  Massimo Bertozzi,et al.  GOLD: a parallel real-time stereo vision system for generic obstacle and lane detection , 1998, IEEE Trans. Image Process..

[45]  Mohan M. Trivedi,et al.  Head Pose Estimation and Augmented Reality Tracking: An Integrated System and Evaluation for Monitoring Driver Awareness , 2010, IEEE Transactions on Intelligent Transportation Systems.

[46]  ZuWhan Kim,et al.  Robust Lane Detection and Tracking in Challenging Scenarios , 2008, IEEE Transactions on Intelligent Transportation Systems.

[47]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.