论文信息 - An Optimization Strategy Based on Hybrid Algorithm of Adam and SGD - 字舞流文

An Optimization Strategy Based on Hybrid Algorithm of Adam and SGD

Yijun Wang | Pengyu Zhou | Wenya Zhong | Yijun Wang | Wenya Zhong | P. Zhou

[1] Jimmy Ba,et al. Adam: A Method for Stochastic Optimization , 2014, ICLR.

[2] Geoffrey E. Hinton,et al. On the importance of initialization and momentum in deep learning , 2013, ICML.

[3] H. Robbins. A Stochastic Approximation Method , 1951 .

[4] Sanjiv Kumar,et al. On the Convergence of Adam and Beyond , 2018 .

[5] Richard Socher,et al. Improving Generalization Performance by Switching from Adam to SGD , 2017, ArXiv.

[6] Yoram Singer,et al. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..