Isotone Optimization in R: Pool-Adjacent-Violators Algorithm (PAVA) and Active Set Methods

In this paper we give a general framework for isotone optimization. First we discuss a generalized version of the Pool-Adjacent-Violators Algorithm (PAVA) to minimize a separable convex function with simple chain constraints. Besides of general convex functions we extend existing PAVA implementations in terms of observation weights, approaches for tie handling, and responses from repeated measurement designs. Since isotone optimization problems can be formulated as convex programming problems with linear constraints we then develop a primal active set method to solve such problem. This methodology is applied on speci�c loss functions relevant in statistics. Both approaches are implemented in the R package isotone.

[1]  H. D. Brunk,et al.  AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION , 1955 .

[2]  H. D. Brunk.  Maximum Likelihood Estimates of Monotone Parameters , 1955 .

[3]  Constance van Eeden,et al.  Testing and estimating ordered parameters of probability distribution , 1958 .

[4]  D. J. Bartholomew,et al.  A TEST OF HOMOGENEITY FOR ORDERED ALTERNATIVES. II , 1959 .

[5]  R. E. Miles.  THE COMPLETE AMALGAMATION INTO BLOCKS, BY WEIGHTED MEANS, OF A FINITE SET OF REAL NUMBERS , 1959 .

[6]  Stephen Warshall,et al.  A Theorem on Boolean Matrices , 1962, JACM.

[7]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[8]  R. Potthoff,et al.  A generalized multivariate analysis of variance model useful especially for growth curve problems , 1964 .

[9]  J. Danskin.  The Theory of Max-Min, with Applications , 1966 .

[10]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[11]  H. D. Brunk,et al.  The Isotonic Regression Problem and its Dual , 1972 .

[12]  J. Kalbfleisch.  Statistical Inference Under Order Restrictions , 1975 .

[13]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[14]  Jan de Leeuw,et al.  Correctness of Kruskal's algorithms for monotone regression with ties , 1977 .

[15]  Richard L. Dykstra.  An isotonic regression algorithm , 1981 .

[16]  Philip E. Gill,et al.  Practical optimization , 1981 .

[17]  R. Dykstra,et al.  An Algorithm for Isotonic Regression for Two or More Independent Variables , 1982 .

[18]  Eliane R. Panier,et al.  An active set method for solving linearly constrained nonsmooth optimization problems , 1987, Math. Program..

[19]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[20]  Michael J. Best,et al.  Active set algorithms for isotonic regression; A unifying framework , 1990, Math. Program..

[21]  Ulf Strömberg,et al.  An algorithm for isotonic regression with arbitrary convex distance function , 1991 .

[22]  Michael J. Best,et al.  Minimizing Separable Convex Functions Subject to Simple Chain Constraints , 1999, SIAM J. Optim..

[23]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[24]  Ravindra K. Ahuja,et al.  A Fast Scaling Algorithm for Minimizing Separable Convex Functions Subject to Chain Constraints , 2001, Oper. Res..

[25]  Oleg Burdakov,et al.  A generalised PAV algorithm for monotonic regression in several variables , 2004 .

[26]  Wei Chu,et al.  Bayesian support vector regression using a unified loss function , 2004, IEEE Transactions on Neural Networks.

[27]  W. Wien.  Econometric Computing with HC and HAC Covariance Matrix Estimators , 2004 .

[28]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[29]  A. Zeileis.  Econometric Computing with HC and HAC Covariance Matrix Estimators , 2004 .

[30]  Kurt Hornik,et al.  A CLUE for CLUster Ensembles , 2005 .

[31]  Editors-in-chief,et al.  Encyclopedia of statistics in behavioral science , 2005 .

[32]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[33]  Jürgen Hansohm.  Algorithms and error estimations for monotone regression on partially preordered sets , 2007 .

[34]  Korbinian Strimmer,et al.  fdrtool: a versatile R package for estimating local and tail area-based false discovery rates , 2008, Bioinform..

[35]  Patrick Mair,et al.  Multidimensional Scaling Using Majorization: SMACOF in R , 2008 .

[36]  C Mittermaier,et al.  Reliability of posturographic measurements in the assessment of impaired sensorimotor function in chronic low back pain. , 2009, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.