An Adaptive Learning Rate for Training Ring-Structured Recurrent Network

A new adaptive learning rate is proposed based on the Lya-punov stability theory for training the Ring-Structured Recurrent Network (RSRN). The adaptive rate is a suucient condition to guarantee the stability and the most rapid convergence of the RSRN dynamic backpropagation algorithm, and it is easily determined in a direct and non-trial manner. Examples of training the RSRN to predict time series are used to demonstrate the eeciency of the learning rate. It has been found that by usage of the adaptive learning rate, the RSRN needs much smaller amount of training time and the resulting network could perform satisfactorily the prediction task.