Non-parametric smoothing for gradient methods in non-differentiable optimization problems

In this article, a method has been established for optimizing multivariate nonlinear discontinuous cost functions having multiple simple kinks in their domains of definition, by applying simple non-parametric inverse trigonometric functions and combinations thereof as smoothing agents. The original function is locally replaced by these smoothing agents at the points of jump discontinuity, while retaining the global structure of the original objective function. The non-parametric smoothing function is exact and devoid of complicated special functions or integral approximations and, hence, the resulting optimization algorithm is relatively simpler and faster. The absence of the parameter ensures little ill-conditioning effects in subsequent calculations. Relevant properties of the smoothing function are developed analytically and the shape of the resulting modified objective functions are illustrated with appropriate numerical simulations. Conjugate gradient and Broyden family of methods (DFP/BFGS) were employed to minimize the modified functions; although Levenberg Marquardt and gradient descent methods were also used occasionally to confirm the universal applicability of the proposed method. Over fifty problems were successfully minimized (maximized) and results for some of them are tabulated before the concluding remarks of the article. Constrained optimization of discontinuous functions with such smoothing agents is an active research within the working group.