Robust obstacle detection and tracking by motion analysis

Within a video-based driver assistance system, it is important to detect and track nearby objects. We describe an approach which evaluates motion information comprised in optical flow vectors. Optical flow contains information about the motion of a camera and about the scene's 3D structure. We use that information to detect obstacles in front of the vehicle. Since the detection is based on motion no a-priori knowledge about obstacle shape is required. Optical flow vectors are estimated from spatio-temporal derivatives of the gray value function which are computed at video frame rate by the hardware MiniVISTA. To eliminate outliers and to speed up obstacle detection the estimated vectors are clustered before they are passed to the obstacle test. The obstacle test separates moving objects from the stationary environment and separates elevated objects from the ground plane. This test is a state estimation problem and enables us to enlarge the motion stereo basis by applying a Kalman filter to track optical flow vectors over subsequent image frames. After 3D grouping of tracked flow vectors of the same obstacle class a state description of each moving object is automatically initialized and updated using a Kalman filter. This algorithm has been implemented on a hardware platform which consists of the system MiniVISTA and two Motorola PowerPCs. This system has been installed in an experimental road vehicle. Results obtained under very different weather conditions show the robustness of the approach.