Experimental Environments for Dismounted Human-Robot Multimodal Communications

The goal for multimodal communication (MMC) is to facilitate the conveyance of information through various modalities, such as auditory, visual, and tactile. MMC has become a major focus for enabling human-robot teaming, but it is often the case that the technological-state of robot capabilities is limited for research and development. Currently, robots often serve a single role, not equipped to interact dynamically with human team members. However, before that functionality is developed, it is important to understand what robot capability is needed for effective collaboration. Through the use of simulations, controlled systematic evaluation of MMC input and output devices can be evaluated to garner a better understanding of how to apply MMC with respect to user’s abilities and preferences, as well as assess the communication hardware and software functionality. An experiment will be presented and discussed to illustrate this approach.

[1]  Florian Jentsch,et al.  From Tools to Teammates , 2011 .

[2]  Norman I. Badler,et al.  Defining Next-Generation Multi-Modal Communication in Human Robot Interaction , 2011 .

[3]  Sharon L. Oviatt,et al.  Advances in Robust Multimodal Interface Design , 2003, IEEE Computer Graphics and Applications.

[4]  Brian Scassellati,et al.  The Oz of Wizard: Simulating the human for interaction research , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[5]  Sharon L. Oviatt,et al.  A rapid semi-automatic simulation technique for investigating interactive speech and handwriting , 1992, ICSLP.

[6]  Laurel D. Riek,et al.  Wizard of Oz studies in HRI , 2012, J. Hum. Robot Interact..

[7]  Jonathan Harris,et al.  Speech and gesture interfaces for squad-level human-robot teaming , 2014, Defense + Security Symposium.

[8]  Sharon L. Oviatt,et al.  Multimodal Interfaces: A Survey of Principles, Models and Frameworks , 2009, Human Machine Interaction.

[9]  Nicu Sebe,et al.  Multimodal Human Computer Interaction: A Survey , 2005, ICCV-HCI.

[10]  J. F. Kelley,et al.  An iterative design methodology for user-friendly natural language office information applications , 1984, TOIS.

[11]  A. Green,et al.  Applying the Wizard-of-Oz framework to cooperative service discovery and configuration , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[12]  Takayuki Kanda,et al.  Group attention control for communication robots with Wizard of OZ approach , 2007, 2007 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI).