Learning Framework for Robust Obstacle Detection, Recognition, and Tracking

This paper introduces a general framework for detection, recognition, and tracking preceding vehicles and pedestrians based on a deep learning approach. The proposed framework combines a novel deep learning approach with the use of multiple sources of local patterns and depth information to yield robust on-road vehicle and pedestrian detection, recognition, and tracking. The proposed system is first based on robust obstacle detection to identify obstacles appearing along the road that are likely to be vehicles and pedestrians, implemented as an efficient adaptive U-V disparity algorithm. Second, the results from the obstacle detection stage are input into a novel vehicle and pedestrian recognition system based on a deep learning model that processes multiple sources of depth information and local patterns. Finally, the results from the recognition stage are used to track detected vehicles or pedestrians in the next frame by means of a proposed tracking and validation model. The proposed framework has been thoroughly evaluated by inputting several vehicle and pedestrian data sets that were collected under various driving conditions. Experimental results show that this framework provides robust vehicle and pedestrian detection, recognition, and tracking with high accuracy, and also satisfies the real-time requirements of driver assistance systems.

[1]  Dariu Gavrila,et al.  A mixed generative-discriminative framework for pedestrian classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Dariu Gavrila,et al.  Ieee Transactions on Intelligent Transportation Systems the Benefits of Dense Stereo for Pedestrian Detection , 2022 .

[3]  Mohan M. Trivedi,et al.  Looking at Vehicles on the Road: A Survey of Vision-Based Vehicle Detection, Tracking, and Behavior Analysis , 2013, IEEE Transactions on Intelligent Transportation Systems.

[4]  Sergiu Nedevschi,et al.  Real-time obstacle detection in complex scenarios using dense stereo vision and optical flow , 2010, 13th International IEEE Conference on Intelligent Transportation Systems.

[5]  Mohan M. Trivedi,et al.  A General Active-Learning Framework for On-Road Vehicle Recognition and Tracking , 2010, IEEE Transactions on Intelligent Transportation Systems.

[6]  Marc'Aurelio Ranzato,et al.  Building high-level features using large scale unsupervised learning , 2011, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Jae Wook Jeon,et al.  A Fast Evolutionary Algorithm for Real-Time Vehicle Detection , 2013, IEEE Transactions on Vehicular Technology.

[8]  Jiri Matas,et al.  A system for real-time detection and tracking of vehicles from a single car-mounted camera , 2012, 2012 15th International IEEE Conference on Intelligent Transportation Systems.

[9]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[10]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[11]  Reinhard Klette,et al.  Performance of Correspondence Algorithms in Vision-Based Driver Assistance Using an Online Image Sequence Database , 2011, IEEE Transactions on Vehicular Technology.

[12]  Alberto Broggi,et al.  Stereo vision-based feature extraction for vehicle detection , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[13]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[14]  Sergiu Nedevschi,et al.  Modeling and Tracking the Driving Environment With a Particle-Based Occupancy Grid , 2011, IEEE Transactions on Intelligent Transportation Systems.

[15]  Bernhard Rinner,et al.  Visual on-line learning in distributed camera networks , 2008, 2008 Second ACM/IEEE International Conference on Distributed Smart Cameras.

[16]  Keiichi Uchimura,et al.  A complete U-V-disparity study for stereovision based 3D driving environment analysis , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[17]  Lei Zhang,et al.  Real-Time Compressive Tracking , 2012, ECCV.

[18]  Hai Jin,et al.  A Map-Reduce Method for Training Autoencoders on Xeon Phi , 2015, 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing.

[19]  Zhenhua Guo,et al.  A Completed Modeling of Local Binary Pattern Operator for Texture Classification , 2010, IEEE Transactions on Image Processing.

[20]  Quoc V. Le,et al.  Tiled convolutional neural networks , 2010, NIPS.

[21]  Alexander Barth,et al.  Moving vehicle detection by optimal segmentation of the Dynamic Stixel World , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[22]  Subrahmanyam Murala,et al.  Local Tetra Patterns: A New Feature Descriptor for Content-Based Image Retrieval , 2012, IEEE Transactions on Image Processing.

[23]  Bernd Jähne,et al.  Outdoor stereo camera system for the generation of real-world benchmark data sets , 2012 .

[24]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Xinxin Zhao,et al.  An obstacle detection algorithm based on U-V disparity map analysis , 2010, 2010 IEEE International Conference on Information Theory and Information Security.

[27]  Dariu Gavrila,et al.  Monocular Pedestrian Detection: Survey and Experiments , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Mayank Bansal,et al.  Real-time vehicle detection for highway driving , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[30]  Ming-Hsuan Yang,et al.  Robust Object Tracking with Online Multiple Instance Learning , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Chung-Cheng Chiu,et al.  Real-Time Front Vehicle Detection Algorithm for an Asynchronous Binocular System , 2010, J. Inf. Sci. Eng..

[32]  Andrew Y. Ng,et al.  Convolutional-Recursive Deep Learning for 3D Object Classification , 2012, NIPS.

[33]  Jae Wook Jeon,et al.  Local Density Encoding for Robust Stereo Matching , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Sergiu Nedevschi,et al.  Tracking multiple objects using particle filters and digital elevation maps , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[35]  Massimo Bertozzi,et al.  Vehicle detection by means of stereo vision-based obstacles features extraction and monocular pattern analysis , 2006, IEEE Transactions on Image Processing.

[36]  Yiannis Aloimonos,et al.  Moving obstacle detection using cameras for driver assistance system , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[37]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.