MultiSEM: A Mulsemedia Model for Supporting the Development of Authoring Tools

This paper proposes MultiSEM (Multimedia Sensory Effect Model), a conceptual model for representing multiple sensory effects in interactive multimedia applications. MultiSEM is an extension of SIMM (Simple Interactive Multimedia Model) and allows the spatiotemporal integration and synchronization of sensory effects with traditional media content. MultiSEM is designed in order to ease the development of mulsemedia authoring tools based on it. To do that, sensory effects are modeled as document nodes, using a similar abstraction as media nodes. The proposed model also aims at supporting the development of authoring tools to be used by those who do not have knowledge of mulsemedia authoring languages and models to create interactive mulsemedia applications. In order to demonstrate that the model can support the development of an authoring tool, MultiSEM is implemented in STEVE 2.0 (Spatio-temporal View Editor), which is an editor for authoring hypermedia applications with sensory effects. Compared with existing mulsemedia models, MultiSEM is the unique model that represents sensory effects as first-class entities and is designed to underlie mulsemedia authoring tools. Besides, an example of mulsemedia application is described and specified using MultiSEM entities. We also present how the application can be created using STEVE 2.0 with sensory effects.