A Domain-Specific Metamodel for Multimedia Processing Systems

In this paper, we introduce 4MPS, a metamodel for multimedia processing systems. The goal of 4MPS is to offer a generic system metamodel that can be instantiated to describe any multimedia processing design. The metamodel combines the advantages of the object-oriented paradigm and metamodeling techniques with system engineering principles and graphical models of computation. 4MPS is based on the classification of multimedia processing objects into two main categories: Processing objects that operate on data and controls, and Data objects that passively hold media content. Processing objects encapsulate a method or algorithm. They also include support for synchronous data processing and asynchronous event-driven Controls as well as a configuration mechanism and an explicit life cycle state model. Data input to and output from Processing objects is done through Ports. Data objects offer a homogeneous interface to media data, and support for metaobject-like facilities such as reflection and serialization. The metamodel can be expressed in the language of graphical models of computation such as the Dataflow Networks and presents a comprehensive conceptual framework for media signal processing applications. 4MPS has its practical validation in several existing environments, including the author's CLAM framework.

[1]  Gilles Kahn,et al.  The Semantics of a Simple Language for Parallel Programming , 1974, IFIP Congress.

[2]  Oz Levia,et al.  Directed Control Dataflow Networks: A New Semantic Model for the System-on-Chip Era , 1999 .

[3]  Lawrence A. Rowe,et al.  Design and performance of the Berkeley Continuous Media Toolkit , 1997, Electronic Imaging.

[4]  Mrinal K. Mandal,et al.  Multimedia Signals and Systems , 2002 .

[5]  George Tzanetakis,et al.  MARSYAS3D: A PROTOTYPE AUDIO BROWSER-EDITOR USING A LARGE SCALE IMMERSIVE VISUAL AND AUDIO DISPLAY , 2001 .

[6]  Stephen Travis Pope,et al.  The Well-Tempered Object: Musical Applications of Object-Oriented Software Technology , 1992 .

[7]  Dennis M. Buede,et al.  The Engineering Design of Systems , 2009 .

[8]  Marian Petre,et al.  Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework , 1996, J. Vis. Lang. Comput..

[9]  Philipp Ackermann,et al.  Direct manipulation of temporal structures in a multimedia application framework , 1994, MULTIMEDIA '94.

[10]  Pau Arumí,et al.  Developing Cross-Platform audio and Music Applications with the CLAM Framework , 2005, ICMC.

[11]  Ed Seidewitz,et al.  What Models Mean , 2003, IEEE Softw..

[12]  Edward A. Lee,et al.  Dataflow process networks , 1995, Proc. IEEE.

[13]  Harald Störrle,et al.  Semantics and Verification of Data Flow in UML 2.0 Activities , 2005, VLFM.

[14]  Anneke Kleppe,et al.  MDA explained - the Model Driven Architecture: practice and promise , 2003, Addison Wesley object technology series.

[15]  George Tzanetakis,et al.  Flexible Scheduling for DataFlow Audio Processing , 2006, ICMC.

[16]  Xavier Amatriain,et al.  A dataflow pattern catalog for sound and music computing , 2006, PLoP '06.

[17]  Jürgen Teich,et al.  SPI - a system model for heterogeneously specified embedded systems , 2002, IEEE Trans. Very Large Scale Integr. Syst..

[18]  Marvin V. Zelkowitz,et al.  Experimental validation in software engineering , 1997, Inf. Softw. Technol..

[19]  Dragos Manolescu,et al.  The Growing Divide in the Patterns World , 2007, IEEE Software.

[20]  Xavier Amatriain.  CLAM: A Framework for Audio and Music Application Development , 2007, IEEE Software.

[21]  David L. Tennenhouse,et al.  The VuSystem: A Programming System for Compute-Intensive Multimedia , 1996, IEEE J. Sel. Areas Commun..

[22]  Heinrich Meyr,et al.  Unified specification of control and data flow , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[23]  Richard Wellershoff.  Multimedia communication systems , 1993 .

[24]  Alexandre R. J. François,et al.  A modular middleware flow scheduling framework (poster session) , 2000, ACM Multimedia.

[25]  Ed F. Deprettere,et al.  Context-aware process networks , 2003, Proceedings IEEE International Conference on Application-Specific Systems, Architectures, and Processors. ASAP 2003.

[26]  Holger Knublauch,et al.  Ontology-Driven Software Development in the Context of the Semantic Web: An Example Scenario with Protégé/OWL , 2004 .

[27]  Stephen J. Mellor,et al.  Model-driven development - Guest editor's introduction , 2003 .

[28]  Stephen Travis Pope,et al.  Metamodels and Design Patterns in CSL4 , 2006, ICMC.

[29]  Pierre de Saqui-Sannes,et al.  Multimedia Authoring with Hierarchical Timed Stream Petri Nets and Java , 2004, Multimedia Tools and Applications.

[30]  David García,et al.  The CLAM Annotator: A Cross-Platform Audio Descriptors Editing Tool , 2005, ISMIR.

[31]  Donald Roberts.  Evolve Frameworks into Domain-Specific Languages , 1997 .

[32]  Steve Kubica,et al.  Cantata: visual programming environment for the Khoros system , 1995, COMG.

[33]  Zoran Bojkovic,et al.  Multimedia Communication Systems: Techniques, Standards, and Networks , 2002 .

[34]  Xavier Amatriain,et al.  An Object-oriented metamodel for digital signal processing with a focus on audio and music , 2005 .

[35]  Hung Keng Pung,et al.  Multimedia Modeling: Modeling Multimedia Information and Systems , 1997 .

[36]  Edward A. Lee,et al.  Overview of the Ptolemy project , 2001 .

[37]  David Garćıa Garzón.  Visual prototyping for audio applications , 2007 .

[38]  Guy E. Garnett,et al.  Virtual digital signal processing in an object-oriented system , 1989 .

[39]  Gabor Karsai,et al.  Self-adaptive software for signal processing , 1998, CACM.

[40]  Alexander Backlund.  The definition of system , 2000 .

[41]  Chandrasekhar Ramakrishnan,et al.  The CREATE Signal Library ("Sizzle"): Design, Issues, and Applications , 2003, ICMC.

[42]  Gregor Engels,et al.  OBJECT-ORIENTED MODELING OF MULTIMEDIA APPLICATIONS , 2002 .

[43]  Gérard G. Medioni,et al.  A modular middleware flow scheduling framework (poster session) , 2000, ACM Multimedia.

[44]  Oscar Pastor,et al.  Model-Driven Development , 2008, Informatik-Spektrum.

[45]  David H. Laidlaw,et al.  The application visualization system: a computational environment for scientific visualization , 1989, IEEE Computer Graphics and Applications.

[46]  Twan Basten,et al.  Requirements on the Execution of Kahn Process Networks , 2003, ESOP.

[47]  Gregor Engels,et al.  Extending UML for modeling of multimedia applications , 1999, Proceedings 1999 IEEE Symposium on Visual Languages.

[48]  David García,et al.  CLAM: a framework for efficient and rapid development of cross-platform audio applications , 2006, ACM Multimedia.

[49]  Tony Clark,et al.  Model-driven development - Guest editor's introduction , 2003 .

[50]  W. D. Rowe.  Why Systems Science and Cybernetics , 1965 .

[51]  Vladan Devedzic,et al.  Understanding ontological engineering , 2002, CACM.

[52]  Perry R. Cook,et al.  Synthesis toolkit in C , 2002, SIGGRAPH 1996.

[53]  Paul Lieverse,et al.  An Overview of Methodologies and Tools in the Field of System-Level Design , 2002, Embedded Processor Design Challenges.

[54]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[55]  Victor Lazzarini.  SOUND PROCESSING WITH THE SndObj LIBRARY: AN OVERVIEW , 2001 .

[56]  Lawrence A. Rowe,et al.  Multimedia systems and applications , 2010, 2010 International Conference on Signal Processing and Multimedia Applications (SIGMAP).

[57]  Tino Lourens,et al.  TiViPE - Tino's Visual Programming Environment , 2004, Proceedings of the 28th Annual International Computer Software and Applications Conference, 2004. COMPSAC 2004..

[58]  George Tzanetakis,et al.  Implicit Patching for Dataflow-Based audio Analysis and synthesis , 2005, ICMC.

[59]  Edward A. Lee,et al.  Static Scheduling of Synchronous Data Flow Programs for Digital Signal Processing , 1989, IEEE Transactions on Computers.

[60]  Gérard G. Medioni,et al.  A Modular Software Architecture for Real-Time Video Processing , 2001, ICVS.

[61]  Cecilia Mascolo,et al.  The SATIN Component System-A Metamodel for Engineering Adaptable Mobile Systems , 2006, IEEE Transactions on Software Engineering.

[62]  Amar Chaudhary,et al.  An Open Architecture for Real-Time Audio Processing Software , 1999 .