A high-performance stereo vision system for obstacle detection

Intelligent vehicle research to date has made great progress toward true autonomy. Integrated systems for on-road vehicles, which include road following, headway maintenance, tactical-level planning, avoidance of large obstacles, and inter-vehicle coordination have been demonstrated. One of the weakest points of current automated cars, however, is the lack of a reliable system to detect small obstacles on the road surface. In order to be useful at highway speeds, such a system must be able to detect small (∼15cm) obstacles at long ranges (∼100m), with a cycle rate of at least 2 Hz. This dissertation presents an obstacle detection system that uses trinocular stereo to detect very small obstacles at long range on highways. The system makes use of the apparent orientation of surfaces in the image in order to determine whether pixels belong to vertical or horizontal surfaces. A simple confidence measure is applied to reject false positives introduced by image noise. The system is capable of detecting objects as small as 14cm high at ranges well in excess of 100m. The obstacle detection system described here relies on several factors. First, the camera system is configured in such a way that even small obstacles generate detectable range measurements. This is done by using a very long baseline, telephoto lenses, and rigid camera mounts. Second, extremely accurate calibration procedures allow accurate determination of these range differences. Multibaseline stereo is used to reduce the number of false matches and to improve range accuracy. Special image filtering techniques are used to enhance the very weak image textures present on the road surface, reducing the number of false range measurements. Finally, a technique for determining the surface orientation directly from stereo data is used to detect the presence of obstacles. A system to detect obstacles is not useful if it does not run in near real-time. In order to improve performance, this dissertation includes a detailed analysis of each stage of the stereo algorithm. An efficient method for rectifying images for trinocular stereo is presented. An analysis of memory usage and cache performance of the stereo matching loop has been performed to allow efficient implementation on systems using general-purpose CPUs. Finally, a method for efficiently determining surface orientation directly from stereo data is described.