On the Examination of the Reliability of Statistical Software for Estimating Logistic Regression Models

The numerical reliability of software packages was examined for the logistic regression model. Software tested include SAS 9.3, MATLAB R2012a, R 3.1.0, Stata/IC 13.1 and LIMDEP 10.5. Thirty benchmark datasets were created by simulating different conditional binary choice processes. To obtain certified values, this study followed the National Institute of Standards and Technology procedures when they generated certified values of parameter estimates and standard errors for the nonlinear logistic regression models used. The logarithm of the relative error was used as a measure of accuracy to examine the numerical reliability of these packages.

[1]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[2]  Bruce D. McCullough,et al.  A Review of RATS v4.2: Benchmarking Numerical Accuracy , 1997 .

[3]  Laszlo Matyas,et al.  Comparative Review of Some Econometric Software Packages , 1993 .

[4]  William G. Tomek.  Confirmation and Replication in Empirical Econometrics: A Step Toward Improved Scholarship , 1993 .

[5]  B. D. McCullough,et al.  On the accuracy of statistical procedures in Microsoft Excel 2003 , 1999 .

[6]  M. Schemper,et al.  A solution to the problem of separation in logistic regression , 2002, Statistics in medicine.

[7]  Colin Rose,et al.  Mathematical Statistics with Mathematica , 2002 .

[8]  Bruce D. McCullough,et al.  Econometric Software Reliability: EViews, LIMDEP, SHAZAM and TSP , 1999 .

[9]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[10]  Micah Altman,et al.  Numerical Issues in Statistical Computing for the Social Scientist , 2003 .

[11]  Robert J. Pavur,et al.  A comparative study of the reliability of nine statistical software packages , 2007, Comput. Stat. Data Anal..

[12]  Bruce D. McCullough,et al.  Is it safe to assume that software is accurate , 2000 .

[13]  Bruce D. McCullough,et al.  The Numerical Reliability of Econometric Software , 1999 .

[14]  Günther Sawitzki,et al.  Report on the numerical reliability of data analysis systems , 1994 .

[15]  Florian Heiss,et al.  Discrete Choice Methods with Simulation , 2016 .

[16]  Jayson L. Lusk,et al.  Mixed logit models: accuracy and software choice , 2011 .

[17]  Marc Nerlove,et al.  On the Numerical Accuracy of Mathematica 5.0 for Doing Linear and Nonlinear Regression , 2005 .

[18]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[19]  B. D. McCullough,et al.  The accurary of Mathematica 4 as a statistical package , 2000, Comput. Stat..

[20]  Allen M. Featherstone,et al.  Reliability of Statistical Software , 2010 .

[21]  Colin R Mckenzie,et al.  2002: A LIMDEP odyssey , 2003 .

[22]  Eberechukwu Onukwugha,et al.  Bernoulli Regression Models: Revisiting the Specification of Statistical Models with Binary Dependent Variables , 2010 .

[23]  Charles G. Renfro,et al.  Some numerical aspects of nonlinear estimation , 2000 .

[24]  W. Greene,et al.  计量经济分析 = Econometric analysis , 2009 .

[25]  Robert A Oster,et al.  An Examination of Statistical Software Packages for Categorical Data Analysis Using Exact Methods—Part II , 2003 .

[26]  Bruce D. McCullough,et al.  Assessing the Reliability of Statistical Software: Part I , 1998 .

[27]  James P. LeSage,et al.  Benchmarking numerical accuracy of statistical algorithms , 1988 .

[28]  Guy Mélard,et al.  On the accuracy of statistical procedures in Microsoft Excel 2010 , 2014, Comput. Stat..

[29]  A comment on ‘Mixed logit models: Accuracy and software choice.' , 2011 .

[30]  H. Vinod,et al.  Verifying the Solution from a Nonlinear Solver: A Case Study , 2004 .

[31]  K. Train,et al.  On the Similarity of Classical and Bayesian Estimates of Individual Mean Partworths , 2000 .

[32]  Bruce D. McCullough,et al.  Wilkinson's Tests and Econometric Software , 1999 .

[33]  Julian Silk,et al.  Systems Estimation: A Comparison of SAS, SHAZAM and TSP , 1996 .

[34]  Houston H. Stokes,et al.  On the advantage of using two or more econometric software systems to solve the same problem , 2004 .

[35]  Robert A. Oster,et al.  An Examination of Statistical Software Packages for Categorical Data Analysis Using Exact Methods , 2002 .

[36]  Stanislav Kolenikov.  Review of Stata 7 , 2001 .

[37]  R. H. Myers.  Generalized Linear Models: With Applications in Engineering and the Sciences , 2001 .

[38]  Michael R. Veall,et al.  Shazam 6.2: A review , 1991 .