On the evaluation of autonomous mobile robots

In this dissertation we pose the problem of evaluating robot performance in a multiple criteria framework. Specifically, we propose a new evaluation methodology for mobile robots navigating in terrain cluttered with obstacles. The technique relies on a combination of multiple statistical criteria. We present a comparative performance analysis of three physical robots to which the evaluation functions were applied. One of the systems constructed for the experiments is a novel quadruped robot; the other two are four wheeled systems. Physics based dynamic simulations of the systems were written, validated, and subjected to the same evaluation functions as the real robots. Nearly 1000 runs of the simulated traverses were performed to obtain a high degree of confidence. Statistics of these runs show that the simulations were good representations of the actual systems. Further, we present sensitivity studies that measure the effects of small parameter changes on the overall system performance. The simulations were also used to study the effects of scale on the mobility metrics. For the navigation task under study we conclude that the robot width and ground clearance are crucial parameters that affect the mobility performance significantly. We also show that the proposed metrics partition the evaluation space in a meaningful way. A diffusion based model was built which predicts robot mobility successfully. The dissertation concludes with a discussion of alternative metrics, alternative techniques to scalarize what is inherently a multidimensional evaluation problem and other directions for future research.