Efficient computation of nonparametric survival functions via a hierarchical mixture formulation

We propose a new algorithm for computing the maximum likelihood estimate of a nonparametric survival function for interval-censored data, by extending the recently-proposed constrained Newton method in a hierarchical fashion. The new algorithm makes use of the fact that a mixture distribution can be recursively written as a mixture of mixtures, and takes a divide-and-conquer approach to break down a large-scale constrained optimization problem into many small-scale ones, which can be solved rapidly. During the course of optimization, the new algorithm, which we call the hierarchical constrained Newton method, can efficiently reallocate the probability mass, both locally and globally, among potential support intervals. Its convergence is theoretically established based on an equilibrium analysis. Numerical study results suggest that the new algorithm is the best choice for data sets of any size and for solutions with any number of support intervals.

[1]  Mike Clarke,et al.  Randomised controlled trials in Africa of HIV and AIDS: descriptive study and spatial distribution , 2005, BMJ : British Medical Journal.

[2]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[3]  Geurt Jongbloed,et al.  Estimating a Unimodal Distribution From Interval-Censored Data , 2006 .

[4]  J. Wellner,et al.  Information Bounds and Nonparametric Maximum Likelihood Estimation , 1992 .

[5]  B. Turnbull The Empirical Distribution Function with Arbitrarily Grouped, Censored, and Truncated Data , 1976 .

[6]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[7]  J. Kalbfleisch,et al.  An Algorithm for Computing the Nonparametric MLE of a Mixing Distribution , 1992 .

[8]  Jianguo Sun,et al.  The Statistical Analysis of Interval-censored Failure Time Data , 2006 .

[9]  A. Dax The smallest point of a polytope , 1990 .

[10]  Dankmar Böhning,et al.  A vertex-exchange-method in D-optimal design theory , 1986 .

[11]  B. Turnbull Nonparametric Estimation of a Survivorship Function with Doubly Censored Data , 1974 .

[12]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[13]  R. Peto,et al.  Experimental Survival Curves for Interval‐Censored Data , 1973 .

[14]  Emmanuel Lesaffre,et al.  A New, Fast Algorithm to Find the Regions of Possible Support for Bivariate Interval-Censored Data , 2004 .

[15]  B. Lindsay Mixture models : theory, geometry, and applications , 1995 .

[16]  Yong Wang On fast computation of the non‐parametric maximum likelihood estimate of a mixing distribution , 2007 .

[17]  June The support reduction algorithm for computing nonparametric function estimates in mixture models June 25 , 2003 , 2022 .

[18]  R. Gentleman,et al.  Computational Algorithms for Censored-Data Problems Using Intersection Graphs , 2001 .

[19]  B. Lindsay,et al.  Alternative EM methods for nonparametric finite mixture models , 2001 .

[20]  Yong Wang,et al.  Dimension-reduced nonparametric maximum likelihood computation for interval-censored data , 2008, Comput. Stat. Data Anal..

[21]  Peter Schlattmann,et al.  Interval censored data: A note on the nonparametric maximum likelihood estimator of the distribution function , 1996 .

[22]  Geurt Jongbloed,et al.  The Iterative Convex Minorant Algorithm for Nonparametric Estimation , 1998 .

[23]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[24]  Qiqing Yu,et al.  Generalized MLE of a Joint Distribution Function with Multivariate Interval-Censored Data , 1999 .

[25]  Changbao Wu,et al.  Some Algorithmic Aspects of the Theory of Optimal Designs , 1978 .

[26]  Jon A. Wellner,et al.  A Hybrid Algorithm for Computation of the Nonparametric Maximum Likelihood Estimator from Censored Data , 1997 .

[27]  Marloes H. Maathuis,et al.  Reduction Algorithm for the NPMLE for the Distribution Function of Bivariate Interval-Censored Data , 2005, 0906.3215.

[28]  L. Mofenson,et al.  Extended antiretroviral prophylaxis to reduce breast-milk HIV-1 transmission. , 2008, The New England journal of medicine.