Attentional multimodal interface for multidrone search in the Alps

We present a multimodal attentional interface suitable for a human operator that monitors and controls the activities of a team of drones during search and rescue missions. We consider a scenario where the operator is a component of the rescue team, hence not fully dedicated to the robots, but only able to interact with them with sparse and incomplete commands. In this context, an adaptive interface is needed to support the user situation awareness and to enable an effective interaction with the drones. In this work, we propose a multimodal attention-based interface designed for this domain. This framework is to filter the information flow towards the operator selecting and adapting the communication mode according to the context and the human state. We illustrate the features of the adaptive system along with an initial assessment in a simulated scenario.

[1]  Steve J. Young,et al.  Cognitive User Interfaces , 2010, IEEE Signal Processing Magazine.

[2]  Silvia Rossi,et al.  An extensible architecture for robust multimodal human-robot communication , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  K L Shapiro,et al.  Temporary suppression of visual processing in an RSVP task: an attentional blink? . , 1992, Journal of experimental psychology. Human perception and performance.

[4]  T. Shallice,et al.  Hierarchical schemas and goals in the control of sequential behavior. , 2006, Psychological review.

[5]  D. Norman,et al.  Attention to Action: Willed and Automatic Control of Behavior Technical Report No. 8006. , 1980 .

[6]  Silvia Rossi,et al.  A dialogue system for multimodal human-robot interaction , 2013, ICMI '13.

[7]  Eric Horvitz,et al.  Display of Information for Time-Critical Decision Making , 1995, UAI.

[8]  Eric Horvitz,et al.  Attention-Sensitive Alerting , 1999, UAI.

[9]  M. Posner,et al.  Attention and the detection of signals. , 1980, Journal of experimental psychology.

[10]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[11]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[12]  Fang Wu,et al.  The economics of attention: maximizing user value in information-rich environments , 2007, ADKDD '07.

[13]  Roland Siegwart,et al.  The SHERPA project: Smart collaboration between humans and ground-aerial robots for improving rescuing activities in alpine environments , 2012, 2012 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[14]  P. Cavanagh,et al.  Independent Resources for Attentional Tracking in the Left and Right Visual Hemifields , 2005, Psychological science.

[15]  Jeffrey S. Shell,et al.  Designing for augmented attention: Towards a framework for attentive user interfaces , 2006, Comput. Hum. Behav..

[16]  William Prinzmetal,et al.  Attention: reaction time and accuracy reveal different mechanisms. , 2005, Journal of experimental psychology. General.

[17]  Silvia Rossi,et al.  Attentional Modulation of Mutually Dependent Behaviors , 2010, SAB.

[18]  Gerhard Tröster,et al.  Discriminating Stress From Cognitive Load Using a Wearable EDA Device , 2010, IEEE Transactions on Information Technology in Biomedicine.

[19]  Alberto Finzi,et al.  Plan Execution and Attentional Regulations for Flexible Human-Robot Interaction , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[20]  Alan D. Baddeley.  Selective attention and performance in dangerous environments. , 1972 .