Image Modeling with Position-Encoding Dynamic Trees

This paper describes the position-encoding dynamic tree (PEDT). The PEDT is a probabilistic model for images that improves on the dynamic tree by allowing the positions of objects to play a part in the model. This increases the flexibility of the model over the dynamic tree and allows the positions of objects to be located and manipulated. This paper motivates and defines this form of probabilistic model using the belief network formalism. A structured variational approach for inference and learning in the PEDT is developed, and the resulting variational updates are obtained, along with additional implementation considerations that ensure the computational cost scales linearly in the number of nodes of the belief network. The PEDT model is demonstrated and compared with the dynamic tree and fixed tree. The structured variational learning method is compared with mean field approaches.

[1]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[2]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[3]  Christopher K. I. Williams,et al.  Combining Belief Networks and Neural Networks for Scene Segmentation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Alan S. Willsky,et al.  Likelihood calculation for a class of multiscale stochastic models, with application to texture discrimination , 1995, IEEE Trans. Image Process..

[5]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[6]  Patrick Pérez,et al.  Noniterative manipulation of discrete energy-based models for image analysis , 1997, Pattern Recognit..

[7]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[8]  Helmut Lucke,et al.  Bayesian Belief Networks as a tool for stochastic parsing , 1995, Speech Commun..

[9]  Michael I. Jordan,et al.  Exploiting Tractable Substructures in Intractable Networks , 1995, NIPS.

[10]  Yee Whye Teh,et al.  Learning to Parse Images , 1999, NIPS.

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  Christopher K. I. Williams,et al.  DTs: Dynamic Trees , 1998, NIPS.

[13]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[14]  Christopher K. I. Williams,et al.  Combining neural networks and belief networks for image segmentation , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[15]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[16]  Azriel Rosenfeld,et al.  Hierarchical Image Analysis Using Irregular Tessellations , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Zoubin Ghahramani,et al.  MFDTs: mean field dynamic trees , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[18]  J. R. Rohlicek,et al.  Parameter estimation of dependence tree models using the EM algorithm , 1995, IEEE Signal Processing Letters.

[19]  Christoph von der Malsburg,et al.  The Correlation Theory of Brain Function , 1994 .

[20]  J. Kittler,et al.  Statistical pattern recognition in image analysis , 1994 .

[21]  Charles A. Bouman,et al.  A multiscale random field model for Bayesian image segmentation , 1994, IEEE Trans. Image Process..

[22]  W. Clem Karl,et al.  Efficient multiscale regularization with applications to the computation of optical flow , 1994, IEEE Trans. Image Process..

[23]  Nicholas J. Adams.  Dynamic Trees: A Hierarchical Probabilistic Approach to Image Modelling , 2001 .

[24]  Thomas S. Huang,et al.  Image processing , 1971 .

[25]  David Heckerman,et al.  Knowledge Representation and Inference in Similarity Networks and Bayesian Multinets , 1996, Artif. Intell..

[26]  Michael I. Jordan.  Graphical Models , 2003 .

[27]  Michèle Basseville,et al.  Modeling and estimation of multiresolution stochastic processes , 1992, IEEE Trans. Inf. Theory.

[28]  Christoph von der Malsburg,et al.  Dynamic link architecture , 1998 .

[29]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[30]  Zoubin Ghahramani,et al.  Propagation Algorithms for Variational Bayesian Learning , 2000, NIPS.

[31]  Paul A. Viola,et al.  A Non-Parametric Multi-Scale Statistical Model for Natural Images , 1997, NIPS.

[32]  Amos J. Storkey.  Dynamic Trees: A Structured Variational Method Giving Efficient Propagation Rules , 2000, UAI.