We present in this article a new technique dedicated to optimise the regularization parameter of a cost function. On the one hand the derivatives of the cost function with regards to the weights permits to optimise the network. On the other the derivatives of the cost function with regards to the regularization parameter permits to optimize the smoothness of the function achieved by the network. We show that by oscillating between these two gradient descent optimisations we achieve the task of regulating the smoothness of a neural network. We present the results of this algorithm on a task design to clearly express the network's level of smoothness.
[1]
Thomas Czernichow.
Architecture Selection through Statistical Sensitivity Analysis
,
1996,
ICANN.
[2]
David E. Rumelhart,et al.
Generalization by Weight-Elimination with Application to Forecasting
,
1990,
NIPS.
[3]
Stephen Jose Hanson,et al.
Minkowski-r Back-Propagation: Learning in Connectionist Models with Non-Euclidian Error Signals
,
1987,
NIPS.
[4]
V. A. Morozov,et al.
Methods for Solving Incorrectly Posed Problems
,
1984
.
[5]
Yves Chauvin.
Dynamic Behavior of Constained Back-Propagation Networks
,
1989,
NIPS.