Humans interacting with multi-robot systems: a natural affect-based approach

This paper proposes a novel human–multi-robot-system interaction approach that enjoys two main features: natural interaction and affect-based adaptation of robots behavior. Specifically, the proposed system enables interaction by means of a wrist-worn device, such as a commercial smartwatch, which allows to track user’s movements and heart activity. Thus, on the one side, the proposed system allows the user to intuitively drive the robots by establishing a natural mapping between wrist movements and robots velocity. On the other side, the system estimates user’s mental fatigue during interaction by means of the analysis of heart rate variability. The proposed interaction system adapts then the behavior of the multi-robot system when the interacting user gets overwhelmed with the interaction and control task, which is then simplified. Experimental validation is provided, to show the effectiveness of the proposed system. First, the natural and affect-based interaction are considered separately. Then, the approach is tested considering a complex realistic scenario, which is simulated in virtual reality in order to get an immersive and realistic interaction experience. The results of the experimental validation clearly show that the proposed affect-based adaptive system leads to relieving the user’s fatigue and mental workload.