Daily HRI evaluation at a classroom environment: reports from dance interaction experiments

The design and development of social robots that interact and assist people in daily life requires moving into unconstrained daily-life environments. This presents unexplored methodological challenges to robotic researchers. Is it possible, for example, to perform useful experiments in the uncontrolled conditions of everyday life environments? How long do these studies need to be to provide reliable results? What evaluations methods can be used?In this paper we present preliminary results on a study designed to evaluate an algorithm for social robots in relatively uncontrolled, daily life conditions. The study was conducted as part of the RUBI project, whose goal is to design and develop social robots by immersion in the environment in which the robots are supposed to operate. First we found that in spite of the relative chaotic conditions and lack of control existing in the daily activities of a child-care center, it is possible to perform experiments in a relatively short period of time and with reliable results. We found that continuous audience response methods borrowed from marketing research provided good inter-observer reliabilities, in the order of 70%, and temporal resolution (the cut-off frequency is in the order of 1 cycle per minute) at low cost (evaluation is performed continuously in real time). We also experimented with objective behavioral descriptions, like tracking children's movement across a room. These approaches complemented each other and provided a useful picture of the temporal dynamics of the child-robot interaction, allowing us to gather baseline data for evaluating future systems. Finally, we also touch the ongoing study of behavior analysis through 3 months long-term child-robot interaction.