A System for Adaptive Multimodal Interaction in Crisis Environments

In the recent years multimodal interfaces have acquired an important role in human computer interaction applications. Subsequently these interfaces become more and more human-oriented. Humans use multimodality to reduce ambiguity and incompleteness of information. Seemingly they are able to switch easily from one modality to the other and fuse the information from different multimodal sources. The goal of our research was to develop a crisis based human like multimodal system. In particular, we bring into focus the multimodal interaction between human users and the automatic crisis system and its correlation with the adaptability to the human behavior in crisis situations. Our system is capable of conceding for an optimal interaction process by taking into account the major informational human channels while gathering the user inputs and producing the system feedback. In this paper we describe the design of our system which is implemented as a running prototype. We have conducted a simulation of a crisis event to measure the degree of user satisfaction. At last we discuss the drawbacks as well as the premises of our solution in the context of the high level of performance achieved by our approach.