Mulsemedia preparation: a new event type for preparing media object presentation and sensory effect rendering

In multimedia applications, a short delay in content delivery may cause synchronization failures among media objects, decreasing the presentation quality and user quality of experience. Moreover, some applications incorporate sensory effects to audiovisual content, which are called mulsemedia applications, imposing even more challenges on application execution control. In order to lessen these problems, document authors or presentation systems can prepare media content and sensory effects in advance. This paper proposes a new operation type over media objects and sensory effects, represented as a new event type in multimedia and mulsemedia models, named preparation event. When sensory effects are modeled as first-class entities, mulsemedia authors may specify relationships that trigger actuator device preparation to ensure that sensory effects will be presented at the correct moment. In order to validate our proposal, some use cases of the preparation event in NCL (Nested Context Language) and Lua applications have been developed and are discussed in the paper. NCL is the standard authoring language for the Brazilian digital TV system declarative middleware, named Ginga-NCL, and also ITU-T H.761 standard for IPTV systems.