An adaptive algorithm for least squares piecewise monotonic data fitting

The number of peaks and troughs of measurements of smooth function values can be unacceptably larger than the number of turning points of the function, when the measurements are too rough. It is proposed to make the least sum of squares change to the data subject to a limit on the number of sign changes of their first divided differences, but usually a suitable value of this limit is not known in advance. It is shown how to obtain automatically an adequate value for it. A test is included that attempts to distinguish between genuine trends and data errors. Specifically, if there are trends, then the monotonic sections of a tentative approximation are increased by one, otherwise this approximation seems to meet the trends and the calculation terminates. The numerical work required per iteration, beyond the second one, is quadratic in the number of data. Details for establishing the underlying algorithm are specified, numerical results from a simulation are included and the test is compared to some well-known residual tests. An application of the algorithm on identifying turning points and trends of data from the Dow Jones stock exchange index is presented. A Fortran implementation of our algorithm provides shorter computation times in practice than the complexity indicates in theory. Further, the single monotonicity problem has found many applications in statistical data analyses within various contexts. More generally, piecewise monotonicity is a property that occurs in a wide range of underlying functions and some important applications of it may be found in detrending data for identifying periodicities (eg. business cycles), or in estimating turning points of a function that is known only by some measurements of its values.

[1]  H. Levene,et al.  On the Power Function of Tests of Randomness Based on Runs up and Down , 1952 .

[2]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[3]  Paul Wintz,et al.  Digital image processing (2nd ed.) , 1987 .

[4]  J. Kalbfleisch.  Statistical Inference Under Order Restrictions , 1975 .

[5]  Paul Dierckx,et al.  Curve and surface fitting with splines , 1994, Monographs on numerical analysis.

[6]  J. Mason,et al.  Algorithms for approximation , 1987 .

[7]  G. Box,et al.  Distribution of Residual Autocorrelations in Autoregressive-Integrated Moving Average Time Series Models , 1970 .

[8]  Jaroslav Hájek,et al.  Theory of rank tests , 1969 .

[9]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[10]  Constance Van Eeden.  Maximum Likelihood Estimation Of Ordered Probabilities1) , 1956 .

[11]  I. C. Demetriou.  Discrete piecewise monotonic approximation by a strictly convex distance function , 1995 .

[12]  W. Härdle.  Applied Nonparametric Regression , 1991 .

[13]  Harold J. Kushner,et al.  A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise , 1964 .

[14]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[15]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[16]  C. Spearman.  The proof and measurement of association between two things. By C. Spearman, 1904. , 1987, The American journal of psychology.

[18]  J. G. Hayes,et al.  Numerical Approximations to Functions and Data. , 1971 .

[19]  Solomos Solomou.  Themes in Macroeconomic History: The UK Economy 1919-1939 , 1996 .

[20]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[21]  M. J. D. Powell,et al.  Least Squares Smoothing of Univariate Data to achieve Piecewise Monotonicity , 1991 .

[22]  G. Wahba.  Spline models for observational data , 1990 .

[23]  W. Edwards,et al.  Decision Analysis and Behavioral Research , 1986 .