Scanning and Sequential Decision Making for Multidimensional Data—Part II: The Noisy Case

We consider the problem of sequential decision making for random fields corrupted by noise. In this scenario, the decision maker observes a noisy version of the data, yet judged with respect to the clean data. In particular, we first consider the problem of scanning and sequentially filtering noisy random fields. In this case, the sequential filter is given the freedom to choose the path over which it traverses the random field (e.g., noisy image or video sequence), thus it is natural to ask what is the best achievable performance and how sensitive this performance is to the choice of the scan. We formally define the problem of scanning and filtering, derive a bound on the best achievable performance, and quantify the excess loss occurring when nonoptimal scanners are used, compared to optimal scanning and filtering. We then discuss the problem of scanning and prediction for noisy random fields. This setting is a natural model for applications such as restoration and coding of noisy images. We formally define the problem of scanning and prediction of a noisy multidimensional array and relate the optimal performance to the clean scandictability defined by Merhav and Weissman. Moreover, bounds on the excess loss due to suboptimal scans are derived, and a universal prediction algorithm is suggested. This paper is the second part of a two-part paper. The first paper dealt with scanning and sequential decision making on noiseless data arrays.

[1]  R. Gray,et al.  Asymptotically Mean Stationary Measures , 1980 .

[2]  László Györfi,et al.  A simple randomized algorithm for sequential prediction of ergodic time series , 1999, IEEE Trans. Inf. Theory.

[3]  Ioannis Kontoyiannis,et al.  Pattern matching and lossy data compression on random fields , 2003, IEEE Trans. Inf. Theory.

[4]  Tamás Linder,et al.  Efficient adaptive algorithms and minimax bounds for zero-delay lossy source coding , 2004, IEEE Transactions on Signal Processing.

[5]  Neri Merhav,et al.  Hidden Markov processes , 2002, IEEE Trans. Inf. Theory.

[6]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[7]  Konstantinos Konstantinides,et al.  Occam filters for stochastic sources with application to digital images , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[8]  Luiz Velho,et al.  Digital halftoning with space filling curves , 1991, SIGGRAPH.

[9]  Tsachy Weissman,et al.  Universal prediction of individual binary sequences in the presence of noise , 2001, IEEE Trans. Inf. Theory.

[10]  Dongning Guo,et al.  Gaussian Channels: Information, Estimation and Multiuser Detection , 2004 .

[11]  Nasir D. Memon,et al.  Lossless Image Compression with a Codebook of Block Scans , 1995, IEEE J. Sel. Areas Commun..

[12]  Abraham Lempel,et al.  Compression of two-dimensional data , 1986, IEEE Trans. Inf. Theory.

[13]  Vladimir Vovk,et al.  Aggregating strategies , 1990, COLT '90.

[14]  T. Duncan ON THE CALCULATION OF MUTUAL INFORMATION , 1970 .

[15]  Tsachy Weissman,et al.  On the optimality of symbol-by-symbol filtering and denoising , 2004, IEEE Transactions on Information Theory.

[16]  Tsachy Weissman,et al.  Scanning and prediction in multidimensional data arrays , 2002, IEEE Trans. Inf. Theory.

[17]  Desh Ranjan,et al.  Space-Filling Curves and Their Use in the Design of Geometric Data Structures , 1997, Theor. Comput. Sci..

[18]  Neri Merhav,et al.  Universal Filtering Via Prediction , 2007, IEEE Transactions on Information Theory.

[19]  Jacob Ziv,et al.  On universal quantization , 1985, IEEE Trans. Inf. Theory.

[20]  Neri Merhav,et al.  Universal prediction of individual sequences , 1992, IEEE Trans. Inf. Theory.

[21]  Neri Merhav,et al.  On sequential strategies for loss functions with memory , 2002, IEEE Trans. Inf. Theory.

[22]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[23]  Kuo-Liang Chung,et al.  Space-filling approach for fast window query on compressed images , 2000, IEEE Trans. Image Process..

[24]  Tsachy Weissman,et al.  Scanning and Sequential Decision Making for Multidimensional Data–Part I: The Noiseless Case , 2007, IEEE Transactions on Information Theory.

[25]  Tsachy Weissman,et al.  Twofold universal prediction schemes for achieving the finite-state predictability of a noisy individual binary sequence , 2001, IEEE Trans. Inf. Theory.

[26]  Tsachy Weissman,et al.  Universal Scanning and Sequential Decision Making for Multidimensional Data , 2006, 2006 IEEE International Symposium on Information Theory.

[27]  Tsachy Weissman,et al.  Universal discrete denoising: known channel , 2003, IEEE Transactions on Information Theory.

[28]  David Haussler,et al.  Sequential Prediction of Individual Sequences Under General Loss Functions , 1998, IEEE Trans. Inf. Theory.

[29]  Ewa Skubalska-Rafajlowicz,et al.  Pattern recognition algorithms based on space-filling curves and orthogonal expansions , 2001, IEEE Trans. Inf. Theory.

[30]  Daniel Cohen-Or,et al.  Context‐based Space Filling Curves , 2000, Comput. Graph. Forum.

[31]  Adam Krzyzak,et al.  Clipped median and space-filling curves in image filtering , 2001 .

[32]  H. Helson,et al.  Prediction theory and Fourier Series in several variables , 1958 .

[33]  Tsachy Weissman,et al.  On limited-delay lossy coding and filtering of individual sequences , 2002, IEEE Trans. Inf. Theory.

[34]  Nelson M. Blachman,et al.  The convolution inequality for entropy powers , 1965, IEEE Trans. Inf. Theory.

[35]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[36]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[37]  Christos Faloutsos,et al.  Analysis of the Clustering Properties of the Hilbert Space-Filling Curve , 2001, IEEE Trans. Knowl. Data Eng..

[38]  Nasir D. Memon,et al.  An analysis of some common scanning techniques for lossless image coding , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[39]  Guillermo Sapiro,et al.  LOCO-I: a low complexity, context-based, lossless image compression algorithm , 1996, Proceedings of Data Compression Conference - DCC '96.

[40]  I. Miller Probability, Random Variables, and Stochastic Processes , 1966 .

[41]  Shlomo Shamai,et al.  Mutual information and minimum mean-square error in Gaussian channels , 2004, IEEE Transactions on Information Theory.

[42]  H. Saunders,et al.  Probability, Random Variables and Stochastic Processes (2nd Edition) , 1989 .

[43]  Amir Dembo,et al.  Source coding, large deviations, and approximate pattern matching , 2001, IEEE Trans. Inf. Theory.

[44]  T. Haijiang,et al.  Lossless image compression via multi-scanning and adaptive linear prediction , 2004, The 2004 IEEE Asia-Pacific Conference on Circuits and Systems, 2004. Proceedings..

[45]  Toby Berger,et al.  Information measures for discrete random fields , 1998 .

[46]  Tsachy Weissman,et al.  Universal prediction of random binary sequences in a noisy environment , 2004 .

[47]  Claude-Henri Lamarque,et al.  Image analysis using space-filling curves and 1D wavelet bases , 1996, Pattern Recognit..

[48]  H. Sagan Space-filling curves , 1994 .

[49]  Tamás Linder,et al.  A zero-delay sequential scheme for lossy coding of individual sequences , 2001, IEEE Trans. Inf. Theory.

[50]  Tsachy Weissman,et al.  The Information Lost in Erasures , 2008, IEEE Transactions on Information Theory.

[51]  Craig Gotsman,et al.  Universal Rendering Sequences for Transparent Vertex Caching of Progressive Meshes , 2002, Comput. Graph. Forum.

[52]  Rolf Niedermeier,et al.  Towards optimal locality in mesh-indexings , 1997, Discret. Appl. Math..