New learning factor and testing methods for conjugate gradient training algorithm

The conjugate gradient method has advantages over backpropagation in the training of artificial neural networks. Unlike previous investigators who have obtained learning factors using computationally expensive iterative line searches, we obtain the optimal learning factor in one step. We validate the learning factor with several tests, and analyze the input bias problem. Examples confirm the usefulness of improved conjugate gradient.

[1]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[2]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[3]  Martin Fodslette Meiller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[4]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[5]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[6]  Farid U. Dowla,et al.  Ship wake-detection procedure using conjugate gradient trained artificial neural networks , 1991, IEEE Trans. Geosci. Remote. Sens..

[7]  Ling Guan,et al.  A recursive approach to joint image restoration and compensated blur identification , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[8]  Michael T. Manry,et al.  Fast training of neural networks for remote sensing , 1994 .

[9]  Ling Guan,et al.  A recursive soft-decision PSF and neural network approach to adaptive blind image regularization , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).

[10]  T. Falas,et al.  Temporal differences learning with the conjugate gradient algorithm , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[11]  Yinghui Xu,et al.  Application of neural networks trained with an improved conjugate gradient algorithm to the turbine fast valving control , 2000, PowerCon 2000. 2000 International Conference on Power System Technology. Proceedings (Cat. No.00EX409).

[12]  Ricardo J. G. B. Campello,et al.  Optimization of hierarchical neural fuzzy models , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[13]  Farid U. Dowla,et al.  Backpropagation Learning for Multilayer Feed-Forward Neural Networks Using the Conjugate Gradient Method , 1991, Int. J. Neural Syst..

[14]  Elizabeth Pattey,et al.  Corn yield prediction with artificial neural network trained using airborne remote sensing and topographic data , 2000, IGARSS 2000. IEEE 2000 International Geoscience and Remote Sensing Symposium. Taking the Pulse of the Planet: The Role of Remote Sensing in Managing the Environment. Proceedings (Cat. No.00CH37120).

[15]  C. Charalambous,et al.  Conjugate gradient algorithm for efficient training of artifi-cial neural networks , 1990 .

[16]  Etienne Barnard,et al.  Optimization for training neural nets , 1992, IEEE Trans. Neural Networks.

[17]  Yuan Baozong,et al.  A fast hybrid algorithm of global optimization for feedforward neural networks , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.