Two-dimensional multi-layer Factor Graphs in Reduced Normal Form

We build a multi-layer architecture using the Bayesian framework of the Factor Graphs in Reduced Normal Form (FGrn). This model allows great modularity and unique localized learning equations. The multi-layer architecture implements a hierarchical data representation that via belief propagation can be used for learning and inference in pattern completion, correction and classification. We apply the framework to images extracted from a standard data set.

[1]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[2]  Fei Wang,et al.  Multilevel Belief Propagation for Fast Inference on Markov Random Fields , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[3]  Robert Nowak,et al.  Multiscale Hidden Markov Models for Bayesian Image Analysis , 1999 .

[4]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Process. Mag..

[5]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[6]  Francesco Palmieri,et al.  Towards Building Deep Networks with Bayesian Factor Graphs , 2015, ArXiv.

[7]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[8]  Francesco Palmieri,et al.  Simulink Implementation of Belief Propagation in Normal Factor Graphs , 2015, Advances in Neural Networks.

[9]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[10]  Charles A. Bouman,et al.  A multiscale random field model for Bayesian image segmentation , 1994, IEEE Trans. Image Process..

[11]  Francesco Palmieri,et al.  A Comparison of Algorithms for Learning Hidden Variables in Normal Graphs , 2013, ArXiv.

[12]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[13]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[14]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[15]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[16]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[17]  A. Willsky.  Multiresolution Markov models for signal and image processing , 2002, Proc. IEEE.

[18]  Honglak Lee,et al.  Sparse deep belief net model for visual area V2 , 2007, NIPS.

[19]  Tengfei Liu,et al.  A Survey on Latent Tree Models and Applications , 2013, J. Artif. Intell. Res..

[20]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[21]  Christian Wolf,et al.  Inference and parameter estimation on hierarchical belief networks for image segmentation , 2010, Neurocomputing.

[22]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[23]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Francesco Palmieri,et al.  Belief propagation and learning in convolution multi-layer factor graphs , 2014, 2014 4th International Workshop on Cognitive Information Processing (CIP).

[25]  Patrick Pérez,et al.  Discrete Markov image modeling and inference on the quadtree , 2000, IEEE Trans. Image Process..

[26]  Andrew Y. Ng,et al.  Learning Feature Representations with K-Means , 2012, Neural Networks: Tricks of the Trade.

[27]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[28]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[29]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[31]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[32]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[33]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[34]  W. Clem Karl,et al.  Multiscale representations of Markov random fields , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[35]  Thomas Serre,et al.  A neuromorphic approach to computer vision , 2010, Commun. ACM.

[36]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[38]  Thomas Hofmann,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2007 .

[39]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[40]  Yoshua Bengio,et al.  On the Expressive Power of Deep Architectures , 2011, ALT.

[41]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[42]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[43]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .