Accelerating Hessian-free optimization for Deep Neural Networks by implicit preconditioning and sampling
暂无分享,去创建一个
Tara N. Sainath | Brian Kingsbury | Bhuvana Ramabhadran | Lior Horesh | Aleksandr Y. Aravkin | T. Sainath | Brian Kingsbury | B. Ramabhadran | A. Aravkin | L. Horesh
[1] J. Nocedal. Updating Quasi-Newton Matrices With Limited Storage , 1980 .
[2] S. Eisenstat. Efficient Implementation of a Class of Preconditioned Conjugate Gradient Methods , 1981 .
[3] J. Shewchuk. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .
[4] Barak A. Pearlmutter. Fast Exact Multiplication by the Hessian , 1994, Neural Computation.
[5] Richard Barrett,et al. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.
[6] Y. Notay. Flexible Conjugate Gradients , 2000, SIAM J. Sci. Comput..
[7] Jorge Nocedal,et al. Automatic Preconditioning by Limited Memory Quasi-Newton Updating , 1999, SIAM J. Optim..
[8] Nicol N. Schraudolph,et al. Fast Curvature Matrix-Vector Products for Second-Order Gradient Descent , 2002, Neural Computation.
[9] D. Holder,et al. Large-Scale Non-Linear 3D Reconstruction Algorithms for Electrical Impedance Tomography of the Human Head , 2007 .
[10] Brian Kingsbury,et al. Lattice-based optimization of sequence classification criteria for neural-network acoustic modeling , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.
[11] James Martens,et al. Deep learning via Hessian-free optimization , 2010, ICML.
[12] Brian Kingsbury,et al. The IBM Attila speech recognition toolkit , 2010, 2010 IEEE Spoken Language Technology Workshop.
[13] Marc'Aurelio Ranzato,et al. Large Scale Distributed Deep Networks , 2012, NIPS.
[14] Daniel Povey,et al. Krylov Subspace Descent for Deep Learning , 2011, AISTATS.
[15] Felix J. Herrmann,et al. Robust inversion, dimensionality reduction, and randomized sampling , 2012, Math. Program..
[16] Tara N. Sainath,et al. Scalable Minimum Bayes Risk Training of Deep Neural Network Acoustic Models Using Distributed Hessian-free Optimization , 2012, INTERSPEECH.
[17] Mark W. Schmidt,et al. Hybrid Deterministic-Stochastic Methods for Data Fitting , 2011, SIAM J. Sci. Comput..
[18] Jorge Nocedal,et al. Sample size selection in optimization methods for machine learning , 2012, Math. Program..
[19] Dong Yu,et al. Error back propagation for sequence training of Context-Dependent Deep NetworkS for conversational speech transcription , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.