论文信息 - Guessing can Outperform Many Long Time Lag Algorithms

Guessing can Outperform Many Long Time Lag Algorithms

Numerous recent papers focus on standard recurrent nets'' inability to deal with long time lags between relevant signals. Some propose rather sophisticated, alternative methods. We show: many problems used to test previous algorithms can be solved more quickly by random weight guessing.

Sepp Hochreiter | Juergen Schmidhuber | S. Hochreiter | J. Schmidhuber

[1] Barak A. Pearlmutter. Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[2] Sepp Hochreiter,et al. Untersuchungen zu dynamischen neuronalen Netzen , 1991 .

[3] Jürgen Schmidhuber,et al. Learning Complex, Extended Sequences Using the Principle of History Compression , 1992, Neural Computation.

[4] Raymond L. Watrous,et al. Induction of Finite-State Languages Using Second-Order Recurrent Networks , 1992, Neural Computation.

[5] Yoshua Bengio,et al. Credit Assignment through Time: Alternatives to Backpropagation , 1993, NIPS.

[6] Yoshua Bengio,et al. An Input Output HMM Architecture , 1994, NIPS.

[7] Yoshua Bengio,et al. Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[8] Panagiotis Manolios,et al. First-Order Recurrent Neural Networks and Deterministic Finite State Automata , 1994, Neural Computation.

[9] Peter Tiňo,et al. Learning long-term dependencies is not as difficult with NARX recurrent neural networks , 1995 .

[10] Corso Elvezia. Discovering Solutions with Low Kolmogorov Complexity and High Generalization Capability , 1995 .

[11] Yoshua Bengio,et al. Hierarchical Recurrent Neural Networks for Long-Term Dependencies , 1995, NIPS.

[12] Jürgen Schmidhuber,et al. Long Short-Term Memory , 1997, Neural Computation.

[13] Jürgen Schmidhuber,et al. Flat Minima , 1997, Neural Computation.