An Approach for Authoring Mulsemedia Documents Based on Events

Although several studies discuss rendering sensory effects in multimedia applications, only a few of them discuss the definition of spatial and temporal synchronization relationships among multimedia content and sensory effects. In order to fill in this gap, this paper proposes an approach for authoring mulsemedia applications based on events. This approach involves the proposal of a mulsemedia conceptual model, called MultiSEM (Multimedia Sensory Effect Model), for supporting the development of graphical authoring environments based on temporal view. The approach also includes the implementation of MultiSEM in a graphical authoring environment, named STEVE 2.0, as proof of concept. STEVE 2.0 is designed for users with few or no knowledge of programming, multimedia/mulsemedia languages and standards. Among existing mulsemedia solutions, our work is unique in that it explores the application level proposing a novel mulsemedia conceptual model.

[1]  Celso A. S. Santos,et al.  Mulsemedia DIY: A Survey of Devices and a Tutorial for Building Your Own Mulsemedia Environment , 2019, ACM Comput. Surv..

[2]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[3]  Débora C. Muchaluat-Saade,et al.  NEXT: graphical editor for authoring NCL documents supporting composite templates , 2013, EuroITV.

[4]  Carlos André Guimarães Ferraz,et al.  MulSeMaker: An MDD Tool for MulSeMedia Web Application Development , 2017, WebMedia.

[5]  Weisi Lin,et al.  Mulsemedia: State of the Art, Perspectives, and Challenges , 2014, TOMM.

[6]  Christian Timmerer,et al.  Improving the Quality of multimedia Experience through sensory effects , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[7]  Gabriel-Miro Muntean,et al.  Beyond Multimedia Adaptation: Quality of Experience-Aware Multi-Sensorial Media Delivery , 2015, IEEE Transactions on Multimedia.

[8]  Kyoungro Yoon,et al.  Streaming Media with Sensory Effect , 2011, 2011 International Conference on Information Science and Applications.

[9]  Christian Timmerer,et al.  An end-to-end tool chain for Sensory Experience based on MPEG-V , 2013, Signal Process. Image Commun..

[10]  Débora C. Muchaluat-Saade,et al.  STEVE: a Hypermedia Authoring Tool based on the Simple Interactive Multimedia Model , 2018, DocEng.

[11]  Luiz Fernando Gomes Soares,et al.  Composer: Authoring Tool for iTV Programs , 2008, EuroITV.

[12]  Débora C. Muchaluat-Saade,et al.  MultiSEM: A Mulsemedia Model for Supporting the Development of Authoring Tools , 2018, WebMedia.

[13]  Luiz Fernando Gomes Soares,et al.  XConnertor and XTemplate: improving the expressiveness and reuse in web authoring languages , 2002, New Rev. Hypermedia Multim..

[14]  Gabriel-Miro Muntean,et al.  Is Multimedia Multisensorial? - A Review of Mulsemedia Systems , 2018, ACM Comput. Surv..

[15]  Rogério Ferreira Rodrigues,et al.  Modeling, authoring and formatting hypermedia documents in the HyperProp system , 2000, Multimedia Systems.

[16]  Débora C. Muchaluat-Saade,et al.  Modeling sensory effects as first-class entities in multimedia applications , 2018, MMSys.

[17]  Christian Timmerer,et al.  Investigating the impact of sensory effects on the Quality of Experience and emotional response in web videos , 2012, 2012 Fourth International Workshop on Quality of Multimedia Experience.

[18]  Sang-Kyun Kim,et al.  Authoring multisensorial content , 2013, Signal Process. Image Commun..

[19]  Fivos Panetsos,et al.  Modeling the Dynamic Behavior of Hypermedia Applications , 2001, IEEE Trans. Software Eng..

[20]  Simone Diniz Junqueira Barbosa,et al.  Extending multimedia languages to support multimodal user interactions , 2017, Multimedia Tools and Applications.

[21]  Ralf Steinmetz,et al.  A Media Synchronization Survey: Reference Model, Specification, and Case Studies , 1996, IEEE J. Sel. Areas Commun..

[22]  Gabriel-Miro Muntean,et al.  Quality of experience assessment of 3D video synchronised with multisensorial media components , 2017, 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB).

[23]  Celso A. S. Santos,et al.  An Event Driven Approach for Integrating Multi-sensory Effects to Interactive Environments , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.