论文信息 - A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning

A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning

A supervised learning algorithm (Scaled Conjugate Gradient, SCG) is introduced TIw pelformance of SCG is benchmarked against that of the standard back propagation algorithm (BP) ( Rumelhart. Hinton. & 14"illiams. 1986 ), the conjugate gradient algorithm with line search ( CGL ) ( Johansson, Dowla. & Goodman, 1990) and the one-step Broyden-Fletcher-Gold./arb-Shanno memoriless quasi-Newton algorithm ( BFGS) ( Battiti, 1990 ). SCG is lhlly-automated, inJudes no critical user-dependent parametepw, and avoids a time consuming line search, which CGL and BFGS use in each iteration in order to determine an appropriate step size. E.¥periments show that SCG is considerablyJhster than BP, CGL, and BFGS.

Martin Fodslette Meiller

[1] M. J. D. Powell,et al. Restart procedures for the conjugate gradient method , 1977, Math. Program..

[2] Magnus R. Hestenes,et al. Conjugate Direction Methods in Optimization , 1980 .

[3] Gerald Tesauro,et al. Scaling Relationships in Back-Propagation Learning: Dependence on Training Set Size , 1987, Complex Syst..

[4] Farid U. Dowla,et al. Backpropagation Learning for Multilayer Feed-Forward Neural Networks Using the Conjugate Gradient Method , 1991, Int. J. Neural Syst..

[5] T. Yoshida,et al. A learning algorithm for multilayered neural networks: a Newton method using automatic differentiation , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[6] Raymond L. Watrous. Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization , 1988 .

[7] Yann LeCun,et al. Generalization and network design strategies , 1989 .

[8] Martin Fodslette Møller,et al. Learning by Conjugate Gradients , 1990, IMYCS.

[9] Roberto Battiti,et al. Accelerated Backpropagation Learning: Two Optimization Methods , 1989, Complex Syst..

[10] Geoffrey E. Hinton. Connectionist Learning Procedures , 1989, Artif. Intell..

[11] Geoffrey E. Hinton,et al. Learning internal representations by error propagation , 1986 .

[12] Roberto Battiti,et al. BFGS Optimization for Faster and Automated Supervised Learning , 1990 .

[13] David G. Luenberger,et al. Linear and nonlinear programming , 1984 .