论文信息 - Studying Convergence of Gradient Algorithms Via Optimal Experimental Design Theory

Studying Convergence of Gradient Algorithms Via Optimal Experimental Design Theory

We study the family of gradient algorithms for solving quadratic optimization problems, where the step-length γ k is chosen according to a particular procedure. To carry out the study, we re-write the algorithms in a normalized form and make a connection with the theory of optimum experimental design. We provide the results of a numerical study which shows that some of the proposed algorithms are extremely efficient.

[1] H. Wynn,et al. Renormalised Steepest Descent in Hilbert Space Converges to a Two-Point Attractor , 2001 .

[2] G. Forsythe. On the asymptotic directions of thes-dimensional optimum gradient method , 1968 .

[3] W. J. Studden,et al. Theory Of Optimal Experiments , 1972 .

[4] Marcos Raydan,et al. Relaxed Steepest Descent and Cauchy-Barzilai-Borwein Method , 2002, Comput. Optim. Appl..

[5] Some remarks on the method of minimal residues , 1982 .

[6] Luc Pronzato,et al. Asymptotic behaviour of a family of gradient algorithms in ℝd and Hilbert spaces , 2006, Math. Program..

[7] H. Akaike. On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method , 1959 .

[8] P. Pardalos,et al. Handbook of global optimization , 1995 .

[9] Luc Pronzato,et al. An Introduction to Dynamical Search , 2002 .

[10] J. Borwein,et al. Two-Point Step Size Gradient Methods , 1988 .

[11] Jorge Nocedal,et al. On the Behavior of the Gradient Norm in the Steepest Descent Method , 2002, Comput. Optim. Appl..