Adaptive Regularization Parameter Selection Method for Enhancing Generalization Capability of Neural Networks

A novel adaptive regularization parameter selection (ARPS) method is proposed in this paper to enhance the performance of the regularization method. The proposed ARPS method enables a gradient descent type training to tunnel through some of the undesired sub-optimal solutions on the composite error surface by means of changing the value of the regularization parameter. Undesired sub-optimal solutions are introduced inherently from regularized objective functions. Hence, the proposed ARPS method is capable of enhancing the regularization method without getting stuck at these sub-optimal solutions.