Detecting trend in randomly switched measurements

We introduce a new trend detection problem inspired by real-time monitoring applications where the origin of the measurements is uncertain: The observed sequence under the alternative hypothesis is the result of a random switching between two sequences, each with a trend. The association between each measurement sample and the two sequences is unknown to the detector. We propose a Generalized Mann-Kendall trend detection algorithm, and show via simulation that it achieves better performance than the Mann-Kendall algorithm for problems with randomly switched measurements. We show that the test statistic can be calculated using an Mixed Integer Linear Programming (MILP) solver. We also show that computing the Generalized Mann-Kendall test statistic can be cast as a Max-Bisection problem, connecting the computation of test statistics to graph optimization.

[1]  H. B. Mann.  Nonparametric Tests Against Trend , 1945 .

[2]  P. Sen.  Estimates of the Regression Coefficient Based on Kendall's Tau , 1968 .

[3]  William W. Melek,et al.  Comparison of trend detection algorithms in the analysis of physiological time-series data , 2005, IEEE Transactions on Biomedical Engineering.

[4]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[5]  A. Morse,et al.  Basic problems in stability and design of switched systems , 1999 .

[6]  Y. Bar-Shalom,et al.  The probabilistic data association filter , 2009, IEEE Control Systems.

[7]  D. Helsel,et al.  Statistical methods in water resources , 2020, Techniques and Methods.

[8]  Robert M. Hirsch,et al.  SELECTION OF METHODS FOR THE DETECTION AND ESTIMATION OF TRENDS IN WATER QUALITY , 1991 .

[9]  R. Hirsch,et al.  A Nonparametric Trend Test for Seasonal Data With Serial Dependence , 1984 .

[10]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[11]  Panos J. Antsaklis,et al.  Optimal control of switched systems based on parameterization of the switching instants , 2004, IEEE Transactions on Automatic Control.

[12]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[13]  E. Dietz,et al.  A Nonparametric Multivariate Test for Monotone Trend with Pharmaceutical Applications , 1981 .

[14]  H. E. Daniels,et al.  Rank Correlation and Population Models , 1950 .

[15]  Maria Eduarda Silva,et al.  Deterministic versus stochastic trends: Detection and challenges , 2009 .

[16]  N. Balakrishnan,et al.  A nonparametric test for trend based on initial ranks , 2006 .

[17]  David S. Johnson,et al.  The NP-Completeness Column: An Ongoing Guide , 1982, J. Algorithms.

[18]  Farzad Farnoud,et al.  An Axiomatic Approach to Constructing Distances for Rank Comparison and Aggregation , 2014, IEEE Transactions on Information Theory.

[19]  Sheng Yue,et al.  A comparison of the power of the t test, Mann-Kendall and bootstrap tests for trend detection / Une comparaison de la puissance des tests t de Student, de Mann-Kendall et du bootstrap pour la détection de tendance , 2004 .

[20]  Hyungbo Shim,et al.  Observability for Switched Linear Systems: Characterization and Observer Design , 2013, IEEE Transactions on Automatic Control.

[21]  Alan M. Frieze,et al.  Improved Approximation Algorithms for MAX k-CUT and MAX BISECTION , 1995, IPCO.

[22]  M. Kendall.  A NEW MEASURE OF RANK CORRELATION , 1938 .

[23]  H. Theil.  A Rank-Invariant Method of Linear and Polynomial Regression Analysis , 1992 .

[24]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[25]  Robin Lougee,et al.  The Common Optimization INterface for Operations Research: Promoting open-source software in the operations research community , 2003, IBM J. Res. Dev..

[26]  Anthony Wirth,et al.  Correlation Clustering , 2010, Encyclopedia of Machine Learning and Data Mining.

[27]  Khaled H. Hamed,et al.  A modified Mann-Kendall trend test for autocorrelated data , 1998 .

[28]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[29]  R. Hirsch,et al.  Techniques of trend analysis for monthly water quality data , 1982 .