Stream Programming for Image and Video Compression

Video playback devices rely on compression algorithms to minimize storage, transmission bandwidth, and overall cost. Compression techniques have high realtime and sustained throughput requirements, and the end of CPU clock scaling means that parallel implementations for novel system architectures are needed. Parallel implementations increase the complexity of application design. Current languages force the programmer to trade off productivity for performance; the performance demands dictate that the parallel programmer choose a low-level language in which he can explicitly control the degree of parallelism and tune his code for performance. This methodology is not cost effective because this architecture-specific code is neither malleable nor portable. Reimplementations must be written from scratch for each of the existing parallel and reconfigurable architectures. This thesis shows that multimedia compression algorithms, composed of many independent processing stages, are a good match for the streaming model of computation. Stream programming models afford certain advantages in terms of programmability, robustness, and achieving high performance. This thesis intends to influence language design towards the inclusion of features that lend to the efficient implementation and parallel execution of streaming applications like image and video compression algorithms. Towards this I contribute i) a clean, malleable, and portable implementation of an MPEG-2 encoder and decoder expressed in a streaming fashion, ii) an analysis of how a streaming language improves programmer productivity, iii) an analysis of how a streaming language enables scalable parallel execution, iv) an enumeration of the language features that are needed to cleanly express compression algorithms, v) an enumeration of the language features that support large scale application development and promote software engineering principles such as portability and reusability. This thesis presents a case study of MPEG-2 encoding and decoding to explicate points about language expressiveness. The work is in the context of the StreamIt programming language.

[1]  Ishfaq Ahmad,et al.  A scalable off-line MPEG-2 video encoding scheme using a multiprocessor system , 2001, Parallel Comput..

[2]  Shuvra S. Bhattacharyya,et al.  Dynamic configuration of dataflow graph topology for DSP system design [video encoder example] , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[3]  Edward A. Lee,et al.  Overview of the Ptolemy project , 2001 .

[4]  Edward A. Lee,et al.  Hierarchical reconfiguration of dataflow models , 2004, Proceedings. Second ACM and IEEE International Conference on Formal Methods and Models for Co-Design, 2004. MEMOCODE '04..

[5]  Henry Hoffmann,et al.  A stream compiler for communication-exposed architectures , 2002, ASPLOS X.

[6]  Ming Lei Liou,et al.  A Data-Parallel Approach for Real-Time MPEG-2 Video Encoding , 1995, J. Parallel Distributed Comput..

[7]  Valentín Valero Ruiz,et al.  Specification and Analysis of the MPEG--2 Video Encoder with Timed-Arc Petri Nets , 2002, Electron. Notes Theor. Comput. Sci..

[8]  Robert J. Safranek,et al.  Image and Video Compression , 1994 .

[9]  William Thies,et al.  StreamIt: A Language for Streaming Applications , 2002, CC.

[10]  Claus Schneider.  Executable specification for multimedia supporting refinement and architecture exploration , 1999, Proceedings 25th EUROMICRO Conference. Informatics: Theory and Practice for the New Millennium.

[11]  Inmos Limited,et al.  OCCAM 2 reference manual , 1988 .

[12]  Calton Pu,et al.  Spidle: A DSL Approach to Specifying Streaming Applications , 2003, GPCE.

[13]  Michael I. Gordon,et al.  Exploiting coarse-grained task, data, and pipeline parallelism in stream programs , 2006, ASPLOS XII.

[14]  Konstantinos Konstantinides,et al.  Image and Video Compression Standards: Algorithms and Architectures , 1997 .

[15]  Kunle Olukotun,et al.  Exploiting Coarse-Grain Parallelism in the MPEG-2 Algorithm , 1998 .

[16]  E. A. de Kock.  Multiprocessor mapping of process networks: a JPEG decoding case study , 2002 .

[17]  Erol Gelenbe,et al.  Image and video compression , 1998 .

[18]  Jim Taylor.  DVD-Video: Multimedia for the Masses , 1999, IEEE Multim..

[19]  M. Balakrishnan,et al.  Exploring design space of parallel realizations: MPEG-2 decoder case study , 2001, Ninth International Symposium on Hardware/Software Codesign. CODES 2001 (IEEE Cat. No.01TH8571).

[20]  Shuvra S. Bhattacharyya,et al.  Modeling of Block-Based DSP Systems , 2005, J. VLSI Signal Process..

[21]  William R. Mark,et al.  Cg: a system for programming graphics hardware in a C-like language , 2003, ACM Trans. Graph..

[22]  Henry Hoffmann,et al.  Evaluation of the Raw microprocessor: an exposed-wire-delay architecture for ILP and streams , 2004, Proceedings. 31st Annual International Symposium on Computer Architecture, 2004..

[23]  Erwin A. de Kock,et al.  YAPI: application modeling for signal processing systems , 2000, Proceedings 37th Design Automation Conference.

[24]  Soonhoi Ha,et al.  Extended Synchronous Dataflow for Efficient DSP System Prototyping , 2002, Des. Autom. Embed. Syst..

[25]  Konstantinos Konstantinides,et al.  Image and video compression standards , 1995 .

[26]  Valérie Bertin,et al.  Modelling, analysis and parallel implementation of an on-line video encoder , 2005, First International Conference on Distributed Frameworks for Multimedia Applications.

[27]  Henry Hoffmann,et al.  The Raw Microprocessor: A Computational Fabric for Software Circuits and General-Purpose Programs , 2002, IEEE Micro.

[28]  Edward A. Lee,et al.  Static Scheduling of Synchronous Data Flow Programs for Digital Signal Processing , 1989, IEEE Transactions on Computers.

[29]  Nicolas Halbwachs,et al.  Programming and verifying critical systems by means of the synchronous data-flow language LUSTRE , 1991 .

[30]  William Thies,et al.  Linear analysis and optimization of stream programs , 2003, PLDI '03.

[31]  Michal Karczmarek,et al.  Constrained and phased scheduling of synchronous data flow graphs for StreamIt language , 2002 .

[32]  Shail Aditya,et al.  Semantics of pH: A parallel dialect of Haskell , 1995 .

[33]  Soonhoi Ha,et al.  Extended Synchronous Dataflow for Efficient DSP System Prototyping , 1999, Proceedings Tenth IEEE International Workshop on Rapid System Prototyping. Shortening the Path from Specification to Prototype (Cat. No.PR00246).

[34]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[35]  Soonhoi Ha,et al.  Efficient dataflow representation of MPEG-1 audio (layer III) decoder algorithm with controlled global states , 1999, 1999 IEEE Workshop on Signal Processing Systems. SiPS 99. Design and Implementation (Cat. No.99TH8461).

[36]  William J. Dally,et al.  Programmable Stream Processors , 2003, Computer.

[37]  Pascal Raymond,et al.  The synchronous data flow programming language LUSTRE , 1991, Proc. IEEE.

[38]  Henry Hoffmann,et al.  MPEG-2 decoding in a stream programming language , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[39]  Miodrag Potkonjak,et al.  MediaBench: a tool for evaluating and synthesizing multimedia and communications systems , 1997, Proceedings of 30th Annual International Symposium on Microarchitecture.

[40]  Edward J. Delp,et al.  Overview of parallel processing approaches to image and video compression , 1994, Electronic Imaging.

[41]  Yen-Kuang Chen,et al.  The ALPBench benchmark suite for complex multimedia applications , 2005, IEEE International. 2005 Proceedings of the IEEE Workload Characterization Symposium, 2005..

[42]  William Thies,et al.  Cache aware optimization of stream programs , 2005, LCTES '05.

[43]  Diego Cazorla,et al.  Specification and Performance of the MPEG-2 Video Encoder by Using the Stochastic Process Algebra: R , 2001 .

[44]  K. Yelick,et al.  Generating Permutation Instructions from a High-Level Description , 2004 .

[45]  Robert Stephens,et al.  A survey of stream processing , 1997, Acta Informatica.

[46]  Pat Hanrahan,et al.  Brook for GPUs: stream computing on graphics hardware , 2004, ACM Trans. Graph..

[47]  William Thies,et al.  Teleport messaging for distributed stream programs , 2005, PPoPP.

[48]  M. Engels,et al.  Grape-II: A System-Level Prototyping Environment for DSP Applications , 1995, Computer.

[49]  William Thies,et al.  Optimizing stream programs using linear state space analysis , 2005, CASES '05.

[50]  L. William Butterworth.  Architecture of the first US direct broadcast satellite system , 1994, Proceedings of IEEE National Telesystems Conference - NTC '94.

[51]  Paolo Ienne,et al.  Parallel Modelling Paradigm in Multimedia Applications: Mapping and Scheduling onto a Multi-Processor System-on-Chip Platform , 2004 .

[52]  Jean-Luc Gaudiot,et al.  The Sisal model of functional programming and its implementation , 1997, Proceedings of IEEE International Symposium on Parallel Algorithms Architecture Synthesis.

[53]  Ishfaq Ahmad,et al.  Performance of software-based MPEG-2 video encoder on parallel and distributed systems , 1997, IEEE Trans. Circuits Syst. Video Technol..

[54]  Paul Le Guernic,et al.  SIGNAL: A declarative language for synchronous programming of real-time systems , 1987, FPCA.

[55]  Gérard Berry,et al.  The Esterel Synchronous Programming Language: Design, Semantics, Implementation , 1992, Sci. Comput. Program..