Measuring user behavior in a complex USAR team evaluation

Performing urban search and rescue (USAR) is a stressful and high demand task. The rescuers can use an extra pair of eyes and ears in the field, especially in places where they themselves cannot go. A ground robot can support rescuers in their task. To develop such a ground robot high fidelity and realism testing is needed [1]. High fidelity evaluations are difficult to set up, because the circumstances need to be complex and truthful. In this paper we performed a complex team experiment in the USAR domain. We measured a large number of variables to assess participants’ performance, task load and emotion. This paper focuses on the question whether the analyses tools (Observer and FaceReader [2]) and measures gave an adequate indication of and insight in rescuers’ operations for a robot with two levels of automation (no versus partial).