On Best Practice Optimization Methods in R

R (R Core Team 2014) provides a powerful and flexible system for statistical computations. It has a default-install set of functionality that can be expanded by the use of several thousand add-in packages as well as user-written scripts. While R is itself a programming language, it has proven relatively easy to incorporate programs in other languages, particularly Fortran and C. Success, however, can lead to its own costs: Users face a confusion of choice when trying to select packages in approaching a problem. A need to maintain workable examples using early methods may mean some tools offered as a default may be dated. In an open-source project like R, how to decide what tools offer "best practice" choices, and how to implement such a policy, present a serious challenge. We discuss these issues with reference to the tools in R for nonlinear parameter estimation (NLPE) and optimization, though for the present article `optimization` will be limited to function minimization of essentially smooth functions with at most bounds constraints on the parameters. We will abbreviate this class of problems as NLPE. We believe that the concepts proposed are transferable to other classes of problems seen by R users.

[1]  John C. Nash.  Functions for Nonlinear Least Squares Solutions , 2014 .

[2]  John C. Nash,et al.  Numerical optimization and surface estimation with imprecise function evaluations , 2003, Stat. Comput..

[3]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[4]  Charles L. Lawson,et al.  Basic Linear Algebra Subprograms for Fortran Usage , 1979, TOMS.

[5]  David Ardia,et al.  DEoptim: An R Package for Global Optimization by Differential Evolution , 2009 .

[6]  Sylvie Huet,et al.  Statistical tools for nonlinear regression : a practical guide with S-PLUS examples , 1997 .

[7]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[8]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[9]  John C. Nash,et al.  Unifying Optimization Algorithms to Aid Software System Users: optimx for R , 2011 .

[10]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[11]  John C. Nash,et al.  Nonlinear parameter estimation: an integrated system in BASIC , 1987 .

[12]  A. D. Hall,et al.  The PORT Mathematical Subroutine Library , 1978, TOMS.

[13]  J. Nash.  Compact Numerical Methods for Computers , 2018 .

[14]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[15]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[16]  John C Nash.  Nonlinear Parameter Optimization Using R Tools , 2014 .