MIDAS-M: A Software Framework for Supporting Multimodal Interaction on Heterogeneous Interaction Devices for Cloud Applications

In this paper, we present a software framework, called MIDAS-M (Mixing and matching heterogeneous Interaction Devices to Applications and Services) that enables an application to lend itself to many different types of interaction methods and accommodate users with different client devices in a flexible manner. In particular, we focus on the aspect of supporting “multimodal” interaction by defining and mapping events that are mixed and matched by different input/output components. The multimodal events defined this way can be realized on various client platforms according to their capabilities. We also describe a case study of applying MIDAS-M to developing a multimodal interface for a virtual apartment preview system, called the SMARTIS, and demonstrate its advantages.