A Review of RATS v4.2: Benchmarking Numerical Accuracy

I have used RATS (Regression Analysis of Time Series; Doan, 1994) as my primary econometrics package for over ten years. In the early days it was extremely frustrating; it was user-hostile, easily accessible only to experienced programmers, the manual was often more confusing than helpful, and it was almost exclusively oriented toward time-series analysis. I persisted nonetheless because it was (and still is, to my knowledge) the only econometrics package which supports frequency domain analysis in more than a purely automatic fashion. The current version bears little resemblance to its predeceedssors: RATS is now user-friendly, well documented, and does much more than time series. However, RATS, like many econometrics packages, makes insufficient effort to assure the user that its results are accurate. Therefore this review will focus on the issue of whether RATS does produce accurate results. Typical software reviews, unfortunately, tend to focus on the transparent, i.e. the userinterface, to the complete exclusion of the latent, i.e. what the computer is actually doing, crunching numbers. This is rather surprising, since we should first inquire whether the program gives an accurate answer, and only then worry about how easy it is to get that answer. Unfortunately, almost all reviews of econometric software completely omit any reference to numerical accuracy, despite the existence of entire collections of benchmarks such as those proposed by Lachenbruck (1983) and Elliott, Reisch, and Campbell (1989). I surveyed three journals which regularly publish software reviews (International Journal of Forecasting, Economic Journal, and Journal of Applied Econometrics) for the years 1990-95. Of more than seventy reviews, only three mentioned numerical accuracy, and only one (Veall, 1991) actually employed a benchmark regression (he used the Lachenbruck tests). This general inattention to numerical accuracy may convey the impression that there are no such problems, when nothing could be farther from the truth: the statistical/econometric software has not been written which does not have numerical deficiencies.