Dimension-reduced nonparametric maximum likelihood computation for interval-censored data

A general technique is proposed for efficient computation of the nonparametric maximum likelihood estimate (NPMLE) of a survival function. The main idea is to include a new support interval that has the largest gradient value between inclusively every two neighbouring support intervals in the support set at each iteration. It is thus able to expand the support set exponentially fast during the initial stage of computation and tends to produce the same support set of the NPMLE afterward. The use of the proposed technique needs to be combined with an algorithm that can effectively find and remove redundant support intervals, for example, the constrained Newton method, the iterative convex minorant algorithm and the subspace-based Newton method. Numerical studies show that the dimension-reducing technique works very well, especially for purely interval-censored data, where a significant computational improvement via dimension reduction is possible. Strengths and weaknesses of various algorithms are also discussed and demonstrated.

[1]  H. D. Brunk,et al.  AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION , 1955 .

[2]  J. Wellner,et al.  Information Bounds and Nonparametric Maximum Likelihood Estimation , 1992 .

[3]  Jon A. Wellner,et al.  Interval censoring, case 2: alternative hypotheses , 1995 .

[4]  Karen H. Haskell,et al.  An algorithm for linear least squares problems with equality and nonnegativity constraints , 1981, Math. Program..

[5]  R. Wolfe,et al.  A semiparametric model for regression analysis of interval-censored failure time data. , 1985, Biometrics.

[6]  Piet Groeneboom,et al.  Lectures on inverse problems , 1996 .

[7]  Dankmar Böhning,et al.  Numerical estimation of a probability measure , 1985 .

[8]  B. Turnbull.  Nonparametric Estimation of a Survivorship Function with Doubly Censored Data , 1974 .

[9]  J. Wellner,et al.  Preservation Theorems for Glivenko-Cantelli and Uniform Glivenko-Cantelli Classes , 2000 .

[10]  Marloes H. Maathuis,et al.  Reduction Algorithm for the NPMLE for the Distribution Function of Bivariate Interval-Censored Data , 2005, 0906.3215.

[11]  B. Turnbull.  The Empirical Distribution Function with Arbitrarily Grouped, Censored, and Truncated Data , 1976 .

[12]  Panos M. Pardalos,et al.  Algorithms for a Class of Isotonic Regression Problems , 1999, Algorithmica.

[13]  C. Witzgall,et al.  Projections onto order simplexes , 1984 .

[14]  Anton Schick,et al.  Consistency of the GMLE with Mixed Case Interval‐Censored Data , 2000 .

[15]  H. Wynn.  The Sequential Generation of $D$-Optimum Experimental Designs , 1970 .

[16]  R. Gentleman,et al.  Computational Algorithms for Censored-Data Problems Using Intersection Graphs , 2001 .

[17]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[18]  H. D. Brunk,et al.  Minimizing integrals in certain classes of monotone functions. , 1957 .

[19]  Geurt Jongbloed,et al.  Estimating a Unimodal Distribution From Interval-Censored Data , 2006 .

[20]  M. Talagrand,et al.  Lectures on Probability Theory and Statistics , 2000 .

[21]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[22]  Qiqing Yu,et al.  Generalized MLE of a Joint Distribution Function with Multivariate Interval-Censored Data , 1999 .

[23]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[24]  Meei Pyng Ng,et al.  A Modification of Peto's Nonparametric Estimation of Survival Curves for Interval‐Censored Data , 2002, Biometrics.

[25]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[26]  Pierre Bernard,et al.  Lectures on Probability Theory and Statistics: Ecole d'Ete de Probabilites de Saint-Flour XXVI - 1996 , 1997 .

[27]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[28]  Michael J. Best,et al.  Active set algorithms for isotonic regression; A unifying framework , 1990, Math. Program..

[29]  Ying Zhang,et al.  On Algorithms for the Nonparametric Maximum Likelihood Estimator of the Failure Function With Censored Data , 2004 .

[30]  Peter Schlattmann,et al.  Interval censored data: A note on the nonparametric maximum likelihood estimator of the distribution function , 1996 .

[31]  Geurt Jongbloed,et al.  The Iterative Convex Minorant Algorithm for Nonparametric Estimation , 1998 .

[32]  R. Peto,et al.  Experimental Survival Curves for Interval‐Censored Data , 1973 .

[33]  J. Kalbfleisch,et al.  An Algorithm for Computing the Nonparametric MLE of a Mixing Distribution , 1992 .

[34]  Jon A. Wellner,et al.  A Hybrid Algorithm for Computation of the Nonparametric Maximum Likelihood Estimator from Censored Data , 1997 .

[35]  H. D. Brunk.  Maximum Likelihood Estimates of Monotone Parameters , 1955 .

[36]  Yong Wang.  On fast computation of the non‐parametric maximum likelihood estimate of a mixing distribution , 2007 .