Survey of Pedestrian Detection for Advanced Driver Assistance Systems

Advanced driver assistance systems (ADASs), and particularly pedestrian protection systems (PPSs), have become an active research area aimed at improving traffic safety. The major challenge of PPSs is the development of reliable on-board pedestrian detection systems. Due to the varying appearance of pedestrians (e.g., different clothes, changing size, aspect ratio, and dynamic shape) and the unstructured environment, it is very difficult to cope with the demanded robustness of this kind of system. Two problems arising in this research area are the lack of public benchmarks and the difficulty in reproducing many of the proposed methods, which makes it difficult to compare the approaches. As a result, surveying the literature by enumerating the proposals one--after-another is not the most useful way to provide a comparative point of view. Accordingly, we present a more convenient strategy to survey the different approaches. We divide the problem of detecting pedestrians from images into different processing steps, each with attached responsibilities. Then, the different proposed methods are analyzed and classified with respect to each processing stage, favoring a comparative viewpoint. Finally, discussion of the important topics is presented, putting special emphasis on the future needs and challenges.

[1]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[2]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[3]  Uwe Franke,et al.  Fast stereo based object detection for stop&go traffic , 1996, Proceedings of Conference on Intelligent Vehicles.

[4]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[5]  Yoram Singer,et al.  Improved Boosting Algorithms Using Confidence-rated Predictions , 1998, COLT' 98.

[6]  Dariu Gavrila,et al.  The Issues , 2011 .

[7]  Liang Zhao,et al.  Stereo- and neural network-based pedestrian detection , 2000, IEEE Trans. Intell. Transp. Syst..

[8]  Uwe Franke,et al.  Real-time stereo vision for urban traffic scene understanding , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[9]  Thorsten Graf,et al.  Flexible object recognition based on invariant theory and agent technology , 2000 .

[10]  J. Friedman Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[11]  Massimo Bertozzi,et al.  Shape-based pedestrian detection , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[12]  Dariu Gavrila,et al.  Pedestrian Detection from a Moving Vehicle , 2000, ECCV.

[13]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[14]  Larry S. Davis,et al.  Pedestrian tracking from a moving vehicle , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[15]  S Milch,et al.  PEDESTRIAN DETECTION WITH RADAR AND COMPUTER VISION , 2001 .

[16]  Tomaso A. Poggio,et al.  Example-Based Object Detection in Images by Components , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Dariu Gavrila,et al.  Sensor-Based Pedestrian Protection , 2001, IEEE Intell. Syst..

[18]  Laurent Itti,et al.  Neuromorphic algorithms for computer vision and attention , 2001, SPIE Optics + Photonics.

[19]  D Lang,et al.  Building safer cars , 2001 .

[20]  Massimo Bertozzi,et al.  Self-calibration of a stereo vision system for automotive applications , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[21]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  Fumio Harashima,et al.  Intelligent Vehicle Technologies , 2002 .

[23]  Jean-Philippe Tarel,et al.  Real time obstacle detection in stereovision on non flat road geometry through "v-disparity" representation , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[24]  Larry S. Davis,et al.  Probabilistic template based pedestrian detection in infrared videos , 2002, Intelligent Vehicle Symposium, 2002. IEEE.

[25]  S. Heinrich,et al.  Fast obstacle detection for urban traffic situations , 2002, IEEE Trans. Intell. Transp. Syst..

[26]  Hiroshi Hattori,et al.  Development of night-vision system , 2002, IEEE Trans. Intell. Transp. Syst..

[27]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Dorin Comaniciu,et al.  An Algorithm for Data-Driven Bandwidth Selection , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Michael Rose,et al.  Stereo-based preprocessing for human shape localization in unstructured environments , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[30]  Shree K. Nayar,et al.  Adaptive dynamic range imaging: optical control of pixel exposures over space and time , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[31]  Sridhar Lakshmanan,et al.  A motion and shape-based pedestrian detection algorithm , 2003, IEEE IV2003 Intelligent Vehicles Symposium. Proceedings (Cat. No.03TH8683).

[32]  J. L. Roux An Introduction to the Kalman Filter , 2003 .

[33]  R. Chapuis,et al.  Shape-based pedestrian detection and localization , 2003, Proceedings of the 2003 IEEE International Conference on Intelligent Transportation Systems.

[34]  Massimo Bertozzi,et al.  IR Pedestrian Detection for Advanced Driver Assistance Systems , 2003, DAGM-Symposium.

[35]  H. Ai,et al.  Boosting nested cascade detector for multi-view face detection , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[36]  Yair Weiss,et al.  Learning object detection from a small number of examples: the importance of good features , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[37]  Keiichi Yamada,et al.  A shape-independent method for pedestrian detection with far-infrared images , 2004, IEEE Transactions on Vehicular Technology.

[38]  A. Shashua,et al.  Pedestrian detection for driving assistance systems: single-frame classification and system level performance , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[39]  B. Schiele,et al.  Combined Object Categorization and Segmentation With an Implicit Shape Model , 2004 .

[40]  Bernt Schiele,et al.  Scale-Invariant Object Categorization Using a Scale-Adaptive Mean-Shift Search , 2004, DAGM-Symposium.

[41]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[42]  T. Dang,et al.  Stereo calibration in vehicles , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[43]  Sergiu Nedevschi,et al.  Stereovision Approach for Obstacle Detection on Non-Planar Roads , 2004, ICINCO.

[44]  D.M. Gavrila,et al.  Vision-based pedestrian detection: the PROTECTOR system , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[45]  W. Ritter,et al.  Reinforcing the reliability of pedestrian detection in far-infrared sensing , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[46]  Dariu Gavrila,et al.  A Bayesian Framework for Multi-cue 3D Object Tracking , 2004, ECCV.

[47]  Hui Sun,et al.  A multi-stage classifier based algorithm of pedestrian detection in night with a near infrared camera in a moving car , 2004, Third International Conference on Image and Graphics (ICIG'04).

[48]  Massimo Bertozzi,et al.  Pedestrian detection for driver assistance using multiresolution infrared vision , 2004, IEEE Transactions on Vehicular Technology.

[49]  Dan Roth,et al.  Learning to detect objects in images via a sparse, part-based representation , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  A. Broggi,et al.  Pedestrian localization and tracking system with Kalman filtering , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[51]  A. Zelinsky,et al.  3D vision sensing for improved pedestrian safety , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[52]  Tomaso A. Poggio,et al.  A Trainable System for Object Detection , 2000, International Journal of Computer Vision.

[53]  Xin Xu,et al.  Logistic Regression and Boosting for Labeled Bags of Instances , 2004, PAKDD.

[54]  M. Mahlisch,et al.  A multiple detector approach to low-resolution FIR pedestrian recognition , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[55]  A. Broggi,et al.  A software video stabilization system for automotive oriented applications , 2005, 2005 IEEE 61st Vehicular Technology Conference.

[56]  Hui Sun,et al.  Nighttime Pedestrian Detection with a Normal Camera Using SVM Classifier , 2005, ISNN.

[57]  Bernt Schiele,et al.  Pedestrian detection in crowded scenes , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[58]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2005, International Journal of Computer Vision.

[59]  M. Szarvas,et al.  Pedestrian detection with convolutional neural networks , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[60]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[61]  A. Broggi,et al.  A modular tracking system for far infrared pedestrian recognition , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[62]  A. Broggi,et al.  Infrared stereo vision-based pedestrian detection , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[63]  G. Wanielik,et al.  Shape and motion-based pedestrian detection in infrared images: a multi sensor approach , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[64]  Takeo Kato,et al.  Pedestrian Detection with Stereo Vision , 2005, 21st International Conference on Data Engineering Workshops (ICDEW'05).

[65]  Z. Hu,et al.  U-V-disparity: an efficient algorithm for stereovision based scene analysis , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[66]  Aude Oliva,et al.  Human Learning of Contextual Priors for Object Search: Where does the time go? , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[67]  L. Andreone,et al.  SVM-based pedestrian recognition on near-infrared images , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[68]  David A. Forsyth,et al.  Computational Studies of Human Motion: Part 1, Tracking and Motion Synthesis , 2005, Found. Trends Comput. Graph. Vis..

[69]  Mohan Trivedi,et al.  Driver activity analysis for intelligent vehicles: issues and development framework , 2005, IEEE Proceedings. Intelligent Vehicles Symposium, 2005..

[70]  K.C.J. Dietmayer,et al.  A pedestrian detection system based on thermopile and radar sensor data fusion , 2005, 2005 7th International Conference on Information Fusion.

[71]  Xia Liu,et al.  Pedestrian detection and tracking with night vision , 2005, IEEE Transactions on Intelligent Transportation Systems.

[72]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[73]  Alberto Broggi,et al.  Model-based validation approaches and matching techniques for automotive vision based pedestrian detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[74]  Mei-Chen Yeh,et al.  Fast Human Detection Using a Cascade of Histograms of Oriented Gradients , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[75]  A. Broggi,et al.  Pedestrian Detection on a Moving Vehicle: an Investigation about Near Infra-Red Images , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[76]  Mohan M. Trivedi,et al.  Multimodal Stereo Image Registration for Pedestrian Detection , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[77]  A. Broggi,et al.  Low-level Pedestrian Detection by means of Visible and Far Infra-red Tetra-vision , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[78]  Alexei A. Efros,et al.  Putting Objects in Perspective , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[79]  Cordelia Schmid,et al.  Human Detection Using Oriented Histograms of Flow and Appearance , 2006, ECCV.

[80]  Tarak Gandhi,et al.  Pedestrian collision avoidance systems: a survey of computer vision based recent studies , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[81]  Navneet Dalal,et al.  Finding People in Images and Videos , 2006 .

[82]  Ramakant Nevatia,et al.  Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors , 2007, International Journal of Computer Vision.

[83]  Dariu Gavrila,et al.  An Experimental Study on Pedestrian Classification , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[84]  A. Broggi,et al.  Pedestrian Detection using Infrared images and Histograms of Oriented Gradients , 2006, 2006 IEEE Intelligent Vehicles Symposium.

[85]  Pietro Cerri,et al.  An evaluation of monocular image stabilization algorithms for automotive applications , 2006, 2006 IEEE Intelligent Transportation Systems Conference.

[86]  Zehang Sun,et al.  On-road vehicle detection: a review , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[87]  Bernt Schiele,et al.  Cross-Articulation Learning for Robust Detection of Pedestrians , 2006, DAGM-Symposium.

[88]  Dariu Gavrila,et al.  Real-time dense stereo for intelligent vehicles , 2006, IEEE Transactions on Intelligent Transportation Systems.

[89]  Dariu Gavrila,et al.  Multi-cue Pedestrian Detection and Tracking from a Moving Vehicle , 2007, International Journal of Computer Vision.

[90]  Joseph Katz,et al.  Pedestrian tracking using thermal infrared imaging , 2006, SPIE Defense + Commercial Sensing.

[91]  Bernt Schiele,et al.  Multi-Aspect Detection of Articulated Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[92]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[93]  James F. O'Brien,et al.  Computational Studies of Human Motion , 2006 .

[94]  A. Broggi,et al.  Pedestrian Detection in Far Infrared Images based on the use of Probabilistic Templates , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[95]  Bernt Schiele,et al.  Robust Object Detection with Interleaved Categorization and Segmentation , 2008, International Journal of Computer Vision.

[96]  Massimo Bertozzi,et al.  A Symmetry-based Validator and Refinement System for Pedestrian Detection in Far Infrared Images , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[97]  Tarak Gandhi,et al.  Pedestrian Protection Systems: Issues, Survey, and Challenges , 2007, IEEE Transactions on Intelligent Transportation Systems.

[98]  Wei Zhang,et al.  Real-time Accurate Object Detection using Multiple Resolutions , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[99]  David A. Forsyth,et al.  Configuration Estimates Improve Pedestrian Finding , 2007, NIPS.

[100]  Giovanni Ramponi,et al.  Video Enhancement and Dynamic Range Control of HDR Sequences for Automotive Applications , 2007, EURASIP J. Adv. Signal Process..

[101]  Angel D. Sappa,et al.  Adaptive Image Sampling and Windows Classification for On-board Pedestrian Detection , 2007 .

[102]  Luc Van Gool,et al.  Depth and Appearance for Mobile Scene Analysis , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[103]  R. Nevatia,et al.  Simultaneous Object Detection and Segmentation by Boosting Local Shape Feature based Classifier , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[104]  W. Ritter,et al.  Detection and Tracking of Multiple Pedestrians in Automotive Applications , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[105]  Luc Van Gool,et al.  Dynamic 3D Scene Analysis from a Moving Vehicle , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[106]  Paulo Peixoto,et al.  A Lidar and Vision-based Approach for Pedestrian and Vehicle Detection and Tracking , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[107]  David Gerónimo Gómez,et al.  Computer Vision Approaches to Pedestrian Detection: Visible Spectrum Survey , 2007, IbPRIA.

[108]  Greg Mori,et al.  Detecting Pedestrians by Learning Shapelet Features , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[109]  Ramakant Nevatia,et al.  Pedestrian Detection in Infrared Images based on Local Shape Features , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[110]  Ignacio Parra,et al.  Combination of Feature Extraction Methods for SVM Pedestrian Detection , 2007, IEEE Transactions on Intelligent Transportation Systems.

[111]  Mohan M. Trivedi,et al.  On Color-, Infrared-, and Multimodal-Stereo Approaches to Pedestrian Detection , 2007, IEEE Transactions on Intelligent Transportation Systems.

[112]  A. Torralba,et al.  The role of context in object recognition , 2007, Trends in Cognitive Sciences.

[113]  Peter Knoll,et al.  HDR Vision for Driver Assistance , 2007 .

[114]  Ramakant Nevatia,et al.  Efficient scan-window based object detection using GPGPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[115]  Rudolf Mester,et al.  Free Space Computation Using Stochastic Occupancy Grids and Dynamic Programming , 2008 .

[116]  Fatih Murat Porikli,et al.  Pedestrian Detection via Classification on Riemannian Manifolds , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[117]  Bernt Schiele,et al.  A Performance Evaluation of Single and Multi-feature People Detection , 2008, DAGM-Symposium.

[118]  Larry S. Davis,et al.  A Pose-Invariant Descriptor for Human Detection and Segmentation , 2008, ECCV.

[119]  Qingming Huang,et al.  Multiple Instance Boost Using Graph Embedding Based Decision Stump for Pedestrian Detection , 2008, ECCV.

[120]  Pietro Perona,et al.  Multiple Component Learning for Object Detection , 2008, ECCV.

[121]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[122]  Luc Van Gool,et al.  A mobile vision system for robust multi-person tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[123]  Joan Serrat,et al.  Nighttime Vehicle Detection for Intelligent Headlight Control , 2008, ACIVS.

[124]  David A. McAllester,et al.  A discriminatively trained, multiscale, deformable part model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[125]  Daniel Snow,et al.  Pedestrian detection using boosted features over many frames , 2008, 2008 19th International Conference on Pattern Recognition.

[126]  Dariu Gavrila,et al.  A mixed generative-discriminative framework for pedestrian classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[127]  Bernt Schiele,et al.  Sliding-Windows for Rapid Object Class Localization: A Parallel Technique , 2008, DAGM-Symposium.

[128]  Subhransu Maji,et al.  Classification using intersection kernel support vector machines is efficient , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[129]  Luc Van Gool,et al.  Articulated Multi-body Tracking under Egomotion , 2008, ECCV.

[130]  Alexei A. Efros,et al.  Closing the loop in scene interpretation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[131]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[132]  Bo Wu,et al.  Pedestrian Tracking by Associating Tracklets using Detection Residuals , 2008, 2008 IEEE Workshop on Motion and video Computing.

[133]  Fadi Dornaika,et al.  An Efficient Approach to Onboard Stereo Vision System Pose Estimation , 2008, IEEE Transactions on Intelligent Transportation Systems.

[134]  B. Schiele,et al.  Pedestrian detection: A benchmark , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[135]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[136]  B. Schiele,et al.  Multi-cue onboard pedestrian detection , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[137]  Kazuhiro Otsuka,et al.  Real-time Visual Tracker by Stream Processing , 2009, J. Signal Process. Syst..

[138]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.