Recurrent Exponential-Family Harmoniums without Backprop-Through-Time

Exponential-family harmoniums (EFHs), which extend restricted Boltzmannmachines (RBMs) from Bernoulli random variables to other exponential families(Welling et al., 2005), are generative models that can be trained withunsupervised-learning techniques, like

[1]  Geoffrey E. Hinton,et al.  Exponential Family Harmoniums with an Application to Information Retrieval , 2004, NIPS.

[2]  Geoffrey E. Hinton,et al.  Parameter estimation for linear dynamical systems , 1996 .

[3]  Geoffrey E. Hinton,et al.  Training Recurrent Neural Networks , 2013 .

[4]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[5]  Joseph G. Makin,et al.  Learning to Estimate Dynamical State with Probabilistic Population Codes , 2015, PLoS Comput. Biol..

[6]  Geoffrey E. Hinton,et al.  Learning Multilevel Distributed Representations for High-Dimensional Sequences , 2007, AISTATS.

[7]  Joseph G. Makin,et al.  Sensory Integration and Density Estimation , 2014, NIPS.

[8]  Jacquelien M.A. Scherpen,et al.  Balanced Realizations, Model Order Reduction, and the Hankel Operator , 2018, The Control Systems Handbook.

[9]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[10]  Joseph G. Makin,et al.  Learning Multisensory Integration and Coordinate Transformation via Density Estimation , 2013, PLoS Comput. Biol..

[11]  C. Matias,et al.  Identifiability of parameters in latent structure models with many observed variables , 2008, 0809.5032.

[12]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[13]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[14]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[15]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[16]  Geoffrey E. Hinton,et al.  The Recurrent Temporal Restricted Boltzmann Machine , 2008, NIPS.