Stereo- and neural network-based pedestrian detection

Pedestrian detection is essential to avoid dangerous traffic situations. We present a fast and robust algorithm for detecting pedestrians in a cluttered scene from a pair of moving cameras. This is achieved through stereo-based segmentation and neural network-based recognition. The algorithm includes three steps. First, we segment the image into sub-image object candidates using disparities discontinuity. Second, we merge and split the sub-image object candidates into sub-images that satisfy pedestrian size and shape constraints. Third, we use intensity gradients of the candidate sub-images as input to a trained neural network for pedestrian recognition. The experiments on a large number of urban street scenes demonstrate that the proposed algorithm: (1) can detect pedestrians in various poses, shapes, sizes, clothing, and occlusion status; (2) runs in real-time; and (3) is robust to illumination and background changes.

[1]  Dean A. Pomerleau,et al.  Knowledge-Based Training of Artificial Neural Networks for Autonomous Robot Driving , 1993 .

[2]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[3]  Robert C. Bolles,et al.  Background modeling for segmentation of video-rate stereo sequences , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[4]  Peter J. Burt,et al.  Object tracking with a moving camera , 1989, [1989] Proceedings. Workshop on Visual Motion.

[5]  Hironobu Fujiyoshi,et al.  Real-time human motion analysis by image skeletonization , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[6]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Charles E. Thorpe,et al.  A new focus for side collision warning systems for transit buses , 2000 .

[8]  Jakub Segen,et al.  A camera-based system for tracking people in real time , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[9]  Osama Masoud,et al.  Robust pedestrian tracking using a model-based approach , 1997, Proceedings of Conference on Intelligent Transportation Systems.

[10]  Larry S. Davis,et al.  W4S : A real-time system for detecting and tracking people in 2 D , 1998, eccv 1998.

[11]  Mahmood Fathy,et al.  Neural-vision based approach for real-time road traffic applications , 1997 .

[12]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[13]  Charles E. Thorpe,et al.  Side collision warning systems for transit buses , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[14]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[15]  John Woodfill,et al.  Real-time stereo vision on the PARTS reconfigurable computer , 1997, Proceedings. The 5th Annual IEEE Symposium on Field-Programmable Custom Computing Machines Cat. No.97TB100186).

[16]  A. Kelly,et al.  Obstacle detection for unmanned ground vehicles: a progress report , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[17]  Henry Dreyfuss,et al.  Measure of Man and Woman: Human Factors in Design , 1993 .

[18]  Theodoros Evgeniou,et al.  A TRAINABLE PEDESTRIAN DETECTION SYSTEM , 1998 .

[19]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[20]  E. Adelson,et al.  Analyzing gait with spatiotemporal surfaces , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[21]  Takeo Kanade,et al.  A stereo machine for video-rate dense depth mapping and its new applications , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  L. Davis,et al.  W 4 S: a Real-time System for Detecting and Tracking People in 2 1 2 D , 1998 .

[23]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Other Conferences.

[24]  C. Wöhler,et al.  A TIME-DELAY NEURAL NETWORK ALGORITHM FOR REAL-TIME PEDESTRIAN RECOGNITION , 1998 .

[25]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[26]  Larry S. Davis,et al.  W4S: A real-time system detecting and tracking people in 2 1/2D , 1998, ECCV.

[27]  Takeo Kanade,et al.  Rotation Invariant Neural Network-Based Face Detection , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[28]  Hideo Mori,et al.  On-line vehicle and pedestrian detections based on sign pattern , 1994, IEEE Trans. Ind. Electron..

[29]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[31]  Gang Xu,et al.  Understanding human motion patterns , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[32]  Tomaso A. Poggio,et al.  Trainable pedestrian detection , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[33]  Dariu Gavrila,et al.  Real-time object detection for "smart" vehicles , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.