Time-triggered static schedulable dataflows for multimedia systems

Software-based reactive multimedia computation systems are pervasive today in desktops but also in mobile and ultra-portable devices. Most such systems offer a callback-based architecture to incorporate specific stream processing. The Synchronous Data flow model (SDF) and its variants are appropriate for many continuous stream processing problems such as the ones involving video and audio. SDF allows for static scheduling of multi-rate processing graphs therefore enabling optimal run-time efficiency. But the SDF abstraction does not adapt well to real-time processing because it lacks the notion of time: executing non-trivial schedules of multi-rate data flows in a time-triggered callback architecture, though possible through buffering, causes jitter, excessive latency and run-time inefficiencies. In this paper we formally describe a new Time-Triggered SDF (TTSDF) model with a static scheduling algorithm that produces periodic schedules than can be split among several callback activations, solving the above-mentioned problems. The model has the same expressiveness than SDF, in the sense that any graph computable by one model will also be computable by the other. Additionally, it enables parallelization and run time load balancing between callback activations.

[1]  Edward A. Lee.  Modeling concurrent real-time processes using discrete events , 1999, Ann. Softw. Eng..

[2]  Walter H. Kohler,et al.  A Preliminary Evaluation of the Critical Path Method for Scheduling Tasks on Multiprocessor Systems , 1975, IEEE Transactions on Computers.

[3]  Nicolas Halbwachs,et al.  Synchronous Programming of Reactive Systems , 1992, CAV.

[4]  Edward A. Lee,et al.  Scheduling dynamic dataflow graphs with bounded memory using the token flow model , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  Gilles Kahn,et al.  Coroutines and Networks of Parallel Processes , 1977, IFIP Congress.

[6]  David García,et al.  A framework for efficient and rapid development of cross-platform audio applications , 2008, MM '06.

[7]  Pau Arumí,et al.  A Data Flow Pattern Language for Audio and Music Computing , 2006 .

[8]  Ross Bencina,et al.  PortAudio - an Open Source Cross Platform Audio API , 2001, ICMC.

[9]  Jack B. Dennis,et al.  First version of a data flow procedure language , 1974, Symposium on Programming.

[10]  Edward A. Lee,et al.  Static Scheduling of Synchronous Data Flow Programs for Digital Signal Processing , 1989, IEEE Transactions on Computers.

[11]  Stephen A. Edwards,et al.  Design of embedded systems: formal models, validation, and synthesis , 1997, Proc. IEEE.

[12]  Edward A. Lee,et al.  Hierarchical finite state machines with multiple concurrency models , 1999, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[13]  Perry R. Cook,et al.  ChucK: a programming language for on-the-fly, real-time audio synthesis and multimedia , 2004, MULTIMEDIA '04.

[14]  Edward A. Lee,et al.  Taming heterogeneity - the Ptolemy approach , 2003, Proc. IEEE.

[15]  Edward A. Lee,et al.  Reinventing Computing for Real Time , 2005, Monterey Workshop.

[16]  Rudy Lauwereins,et al.  Data memory minimisation for synchronous data flow graphs emulated on DSP-FPGA targets , 1997, DAC.

[17]  Carlo Rossi,et al.  Giotto a time-triggered language for embedded programming , 2011 .

[18]  Twan Basten,et al.  Requirements on the Execution of Kahn Process Networks , 2003, ESOP.

[19]  Peter H. N. de With,et al.  Data-flow timing models of dynamic multimedia applications for multiprocessor systems , 2004, 4th IEEE International Workshop on System-on-Chip for Real-Time Applications.

[20]  George Tzanetakis,et al.  MARSYAS-0.2: A Case Study in Implementing Music Information Retrieval Systems , 2008 .

[21]  A. Kerihuel,et al.  VSDF: synchronous data flow for VLSI , 1994, Proceedings of 1994 37th Midwest Symposium on Circuits and Systems.

[22]  Edward A. Lee,et al.  Dataflow process networks , 2001 .