An experimental analysis of classifier ensembles for learning drifting concepts over time in autonomous outdoor robot navigation

Autonomous robot navigation in unstructured outdoor environments is a challenging area of active research and is currently unsolved. The navigation task requires identifying safe, traversable paths which allow the robot to progress toward a goal while avoiding obstacles. Stereo is an effective tool in the near field, but for smooth long-range trajectory planning or fast driving an approach is needed to understand far-field terrain as well. One approach is to apply Machine Learning techniques that accomplish near-to-far learning by augmenting near-field Stereo readings with learned classifications of the appearance of safe terrain and obstacles in the far field. A key problem with basic instantiations of this approach is that they are not able to identify obstacles in the far field unless there are examples of those obstacles in the near field within the same image, which is not always the case. This leads to a common failure mode in autonomous navigation where incorrect trajectories are followed by the robot as a result of short-sightedness. This thesis proposes to address this problem through the use of classifier ensembles which serve as a mechanism to store previously learned terrain models. Such ensembles are shown in the literature to improve predictive performance in both static environments and in the dynamic environments associated with the problem domain. In this domain, individual models are trained over time and added to an on-line model library as the robot navigates towards a goal. These stored models serve as memory and can be used for terrain classification of an incoming image. The key issues for this task are model selection of appropriate models from the library and the subsequent model combination of each selected model's output. Several methods for selecting and combining models from the library are proposed: choosing the best K models from the library, Bayesian Model Averaging, and a proposed adaptation of Ensemble Selection for use in dynamic environments. An extensive experimental evaluation is performed. This analysis is conducted on novel hand-labeled datasets taken directly from the problem domain, which are shown to contain drifting concepts. Several baselines are considered, including the one-model-per-image approach used in the most basic near-to-far learning. The experimental results uncover many important differences in the behavior of ensemble methods versus simpler approaches. In many scenarios, it is shown that the use of classifier ensembles increases far-field predictive performance compared to non-ensemble approaches, although this result was not shown to be statistically significant over all datasets and metrics in the analysis.