Source coding with channel, distortion, and complexity constraints

Data compression methods have progressed from coding a simple alphabet according to known probabilities to a multistage process involving breaking down complex data structures, estimating conditional probabilities, and producing an output data stream. A variety of methods exist for converting alphabet symbols with known or estimated probabilities into a compressed bit stream. Arithmetic coding is one method that allows great efficiency and generality for describing the data. This dissertation presents a generalization of arithmetic coding which allows data compression to be performed onto constrained channels. This is a natural endeavor since compressed data must be stored or transmitted and some constraints typically exist on these processes. This generalization allows the compressed data stream to automatically meet constraints such as a run length limited constraint, a trellis code error correction constraint, or even a Morse code channel constraint. In addition, a simple implementation is presented using finite state machines. Results show that under certain circumstances simple finite state machines can perform compression as efficiently as the QM-coder used in the JPEG and JBIG international standards. In addition, discrete channel modulation can be performed more efficiently than currently employed fixed rate encoders, although error propagation is increased. Joint compression and modulation operations can be more efficient and less complex than separate operations. In this dissertation probability estimation and modeling aspects of a data compression system are discussed. Transform coders are analyzed in some detail, leading to a prediction of bit rate and distortion for a class of encoders including the JPEG standard. A three-way trade-off between computation and bit rate and distortion is investigated. Lastly, a brief examination of minimum entropy quantization is made in the context of images compression with a maximum error criterion.