Computationally Efficient Stochastic Realization for Internal Multiscale Autoregressive Models

In this paper we develop a stochastic realization theory for multiscale autoregressive (MAR) processes that leads to computationally efficient realization algorithms. The utility of MAR processes has been limited by the fact that the previously known general purpose realization algorithm, based on canonical correlations, leads to model inconsistencies and has complexity quartic in problem size. Our realization theory and algorithms addresses these issues by focusing on the estimation-theoretic concept of predictive efficiency and by exploiting the scale-recursive structure of so-called internal MAR processes. Our realization algorithm has complexity quadratic in problem size and with an approximation we also obtain an algorithm that has complexity linear in problem size.

[1]  Alan S. Willsky,et al.  Likelihood calculation for a class of multiscale stochastic models, with application to texture discrimination , 1995, IEEE Trans. Image Process..

[2]  K. Arun,et al.  Balanced approximation of stochastic systems , 1990 .

[3]  Alan S. Willsky,et al.  A canonical correlations approach to multiscale stochastic realization , 2001, IEEE Trans. Autom. Control..

[4]  Carl Wunsch,et al.  Mapping Mediterranean Altimeter Data with a Multiresolution Optimal Interpolation Algorithm , 1998 .

[5]  Michael I. Jordan.  Graphical Models , 2003 .

[6]  Calyampudi R. Rao.  The use and interpretation of principal component analysis in applied research , 1964 .

[7]  W. Clem Karl,et al.  Multiscale representations of Markov random fields , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8]  Paul W. Fieguth,et al.  Multiresolution stochastic models for the efficient solution of large-scale space-time estimation problems , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[9]  A. Willsky,et al.  A multiresolution methodology for signal-level fusion and data assimilation with applications to remote sensing , 1997, Proc. IEEE.

[10]  K. C. Chou,et al.  Multiscale systems, Kalman filters, and Riccati equations , 1994, IEEE Trans. Autom. Control..

[11]  Alan S. Willsky,et al.  The Modeling and Estimation of Statistically Self-Similar Processes in a Multiresolution Framework , 1999, IEEE Trans. Inf. Theory.

[12]  W. W. Irving,et al.  A multiresolution approach to discrimination in SAR imagery , 1997, IEEE Transactions on Aerospace and Electronic Systems.

[13]  Sanjeev Khudanpur,et al.  Tree-structured models of parameter dependence for rapid adaptation in large vocabulary conversational speech recognition , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[14]  C. R. Dietrich,et al.  Fast and Exact Simulation of Stationary Gaussian Processes through Circulant Embedding of the Covariance Matrix , 1997, SIAM J. Sci. Comput..

[15]  Alan S. Willsky,et al.  Multiresolution approach to discriminating targets from clutter in SAR imagery , 1995, Defense, Security, and Sensing.

[16]  Praveen Kumar,et al.  Basin level statistical properties of topographic index for North America , 2000 .

[17]  B. Mandelbrot,et al.  Fractional Brownian Motions, Fractional Noises and Applications , 1968 .

[18]  W. Clem Karl,et al.  Efficient multiscale regularization with applications to the computation of optical flow , 1994, IEEE Trans. Image Process..

[19]  W. W. Irving,et al.  Multiscale stochastic realization and model identification with applications to large-scale estimation problems , 1995 .

[20]  Austin B. Frakt.  Internal multiscale autoregressive processes, stochastic realization, and covariance extension , 1999 .

[21]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[22]  W. C. Karl,et al.  Eecient Multiscale Regularization with Applications to the Computation of Optical Flow 1 , 1993 .

[23]  H. Akaike.  Stochastic theory of minimal realization , 1974 .

[24]  Praveen Kumar,et al.  A multiple scale state-space model for characterizing subgrid scale variability of near-surface soil moisture , 1999, IEEE Trans. Geosci. Remote. Sens..

[25]  Paul W. Fieguth,et al.  Multiresolution optimal interpolation and statistical analysis of TOPEX/POSEIDON satellite altimetry , 1995, IEEE Transactions on Geoscience and Remote Sensing.

[26]  Michael M. Daniel,et al.  Multiresolution statistical modeling with application to modeling groundwater flow , 1997 .

[27]  Paul W. Fieguth,et al.  Multiscale methods for the segmentation and reconstruction of signals and images , 2000, IEEE Trans. Image Process..

[28]  A. Willsky,et al.  A multiscale approach for estimating solute travel time distributions , 2000 .

[29]  W. Clem Karl,et al.  Multiscale representations of Markov random fields , 1993, IEEE Trans. Signal Process..

[30]  Alan S. Willsky,et al.  A theory for multiscale stochastic realization , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[31]  Ashvin Kannan.  Adaptation of spectral trajectory models for large vocabulary continuous speech recognition , 1997 .

[32]  Paul W. Fieguth,et al.  Efficient Multiresolution Counterparts to Variational Methods for Surface Reconstruction , 1998, Comput. Vis. Image Underst..

[33]  Mari Ostendorf,et al.  Modeling dependency in adaptation of acoustic models using multiscale tree processes , 1997, EUROSPEECH.

[34]  H. Krim,et al.  Hierarchical stochastic modeling of SAR imagery for segmentation/compression , 1999, IEEE Trans. Signal Process..

[35]  W. Clem Karl,et al.  Multiscale segmentation and anomaly enhancement of SAR imagery , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[36]  Paul W. Fieguth,et al.  Fractal estimation using models on multiscale trees , 1996, IEEE Trans. Signal Process..

[37]  K. C. Chou,et al.  Multiscale recursive estimation, data fusion, and regularization , 1994, IEEE Trans. Autom. Control..

[38]  Paul W. Fieguth,et al.  An overlapping tree approach to multiscale stochastic modeling and estimation , 1997, IEEE Trans. Image Process..

[39]  H. Akaike.  Markovian Representation of Stochastic Processes by Canonical Variables , 1975 .

[40]  Paul W. Fieguth,et al.  Multiscale methods for the segmentation of images , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[41]  Rekha Govil,et al.  Neural Networks in Signal Processing , 2000 .

[42]  G. Picci,et al.  On the Stochastic Realization Problem , 1979 .

[43]  Alan S. Willsky,et al.  Multiscale Autoregressive Models and Wavelets , 1999, IEEE Trans. Inf. Theory.

[44]  Jan Beran,et al.  Statistics for long-memory processes , 1994 .

[45]  Marco Alfaro Sironvalle.  The random coin method: Solution of the problem of the simulation of a random function in the plane , 1980 .