On Optimization Algorithms for Maximum Likelihood Estimation

Maximum likelihood estimation (MLE) is one of the most popular technique in econometric and other statistical applications due to its strong theoretical appeal, but can lead to numerical issues when the underlying optimization problem is solved. We examine in this paper a range of trust region and line search algorithms and focus on the impact that the approximation of the Hessian matrix has on their respective performance. In particular, we propose new methods to switch between the approximation schemes and compare the effectiveness of these strategies with existing approaches. We assess the numerical efficiency of the proposed switching methods for the estimation of discrete choice models, more precisely mixed logit and logit based route choice models.

[1]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[2]  Emma Frejinger,et al.  Random Sampling of Alternatives in a Route Choice Context , 2007 .

[3]  D. McFadden,et al.  MIXED MNL MODELS FOR DISCRETE RESPONSE , 2000 .

[4]  Philippe L. Toint,et al.  Estimating Nonparametric Random Utility Models with an Application to the Value of Time in Heterogeneous Populations , 2010, Transp. Sci..

[5]  Moshe Ben-Akiva,et al.  Discrete Choice Analysis: Theory and Application to Travel Demand , 1985 .

[6]  Tien Mai,et al.  A misspecification test for logit based route choice models , 2015 .

[7]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[8]  P. L’Ecuyer,et al.  Estimation of the mixed logit likelihood function by randomized quasi-Monte Carlo , 2010 .

[9]  Larry Nazareth,et al.  A family of variable metric updates , 1977, Math. Program..

[10]  Florian Heiss,et al.  Discrete Choice Methods with Simulation , 2016 .

[11]  Shlomo Bekhor,et al.  Link-Nested Logit Model of Route Choice: Overcoming Route Overlapping Problem , 1998 .

[12]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[13]  Daniel McFadden,et al.  Modelling the Choice of Residential Location , 1977 .

[14]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[15]  Philippe L. Toint,et al.  An adaptive Monte Carlo algorithm for computing mixed logit estimators , 2006, Comput. Manag. Sci..

[16]  D. McFadden.  Econometric Models of Probabilistic Choice , 1981 .

[17]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[18]  Bronwyn H Hall,et al.  Estimation and Inference in Nonlinear Structural Models , 1974 .

[19]  M. Bierlaire,et al.  Sampling of Alternatives for Route Choice Modeling , 2009 .

[20]  David S. Bunch.  Maximum likelihood estimation of probabilistic choice methods , 1987 .

[21]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[22]  W. Newey,et al.  Large sample estimation and hypothesis testing , 1986 .

[23]  Anders Karlström,et al.  A link based network route choice model with unrestricted choice set , 2013 .

[24]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[25]  S. Ilker Birbil,et al.  A symmetric rank-one quasi-Newton line-search method using negative curvature directions , 2011, Optim. Methods Softw..

[26]  Nicholas I. M. Gould,et al.  Convergence of quasi-Newton matrices generated by the symmetric rank one update , 1991, Math. Program..

[27]  John E. Dennis,et al.  An Adaptive Nonlinear Least-Squares Algorithm , 1977, TOMS.

[28]  M. Bierlaire,et al.  Discrete Choice Methods and their Applications to Short Term Travel Decisions , 1999 .

[29]  D. McFadden.  Conditional logit analysis of qualitative choice behavior , 1972 .