Behaviour classification for temporal data with limited readings

Classifying items using temporal data, i.e. several readings of the same attribute in different time points, has many applications in the real world. The pivotal question which motivates this study is: ''Is it possible to quantify behavioural change in temporal data? And what is the best reference point to compare the behaviour change with?". The focus of this study will be in applications in economics such as playing many rounds of public goods games and share price moves in the stock market. There are many methods for classifying temporal data and many methods for measuring the change of items' behaviour in temporal data. However, the available methods for classifying temporal data produce complicated rules, and their models are buried in deep decision trees or complex neural networks that are hard for human experts to read and understand. Moreover, methods of measuring cluster changes do not focus on the individual item's behaviour rather; they concentrate on the clusters and their changes over time. This research presents methods for classifying temporal data items and measuring their behavioural changes between time points. As case of studies, public goods game and stock market price data are used to test novel methods of classification and behaviour change measure. To represent the magnitude of the behaviour change, we use cluster validity measures in a novel way by measuring the difference between item labels produced by the same clustering algorithm at each time point and a behaviour reference point. Such a reference point might be the first time point, the previous time point or a point representing the general overall behaviour of the items in the temporal data. This method uses external cluster validity indices to measure the difference between labels provided by the same clustering method in different time points rather than using different clustering methods for the same data set as it is the case for relative clustering indices. To create a general behavioural reference point in temporal data, we present a novel temporal rule-based classification method that consists of two stages. In the first stage, initial rules are generated based on experts' definition for the classes in the form of aggregated attributes of the temporal readings. These initial rules are not crisp and may overlap in their representation for the classes. This provides flexibility for the rules so that they can create a pool of classifiers that can be selected from. Then this pool of classifiers will be optimised in the second stage so that an optimised classifier will be selected among them. The optimised classifier is a set of discrete classification rules, which generates the most compact classes over all time points. Class compactness is measured by using statistical dispersion measures or Euclidean distance within class items. The classification results of the public goods game show that the proposed method for classification can produce better results for representing players than the available methods by economists and general temporal classification methods. Moreover, measuring players' behaviour supports economists' view of the players' behaviour change during game rounds. For the stock market data, we present a viable method for classifying stocks according to their stability which might help to provide insights for stock market predictability.

[1]  Eamonn J. Keogh,et al.  Scaling up dynamic time warping for datamining applications , 2000, KDD '00.

[2]  Lotfi A. Zadeh,et al.  MAKING COMPUTERS THINK LIKE PEOPLE , 1984 .

[3]  Michalis Vazirgiannis,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques , 2022 .

[4]  Myra Spiliopoulou,et al.  MONIC and Followups on Modeling and Monitoring Cluster Transitions , 2013, ECML/PKDD.

[5]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[6]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Vladimir Estivill-Castro,et al.  Why so many clustering algorithms: a position paper , 2002, SKDD.

[8]  N. Shephard,et al.  Multivariate stochastic variance models , 1994 .

[9]  R. Lawrence Rule-Based Classification Systems Using Classification and Regression Tree (CART) Analysis , 2001 .

[10]  Cuntai Guan,et al.  Temporal classification of multichannel near-infrared spectroscopy signals of motor imagery for developing a brain–computer interface , 2007, NeuroImage.

[11]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[12]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[13]  Robi Polikar,et al.  Incremental Learning of Concept Drift in Nonstationary Environments , 2011, IEEE Transactions on Neural Networks.

[14]  Matthew O. Ward,et al.  CLUES: a unified framework supporting interactive exploration of density-based clusters in streams , 2011, CIKM '11.

[15]  Li Xiaofeng,et al.  Study on a Classification Model of Data Stream based on Concept Drift , 2014, MUE 2014.

[16]  Andrew P. Bradley,et al.  The use of the area under the ROC curve in the evaluation of machine learning algorithms , 1997, Pattern Recognit..

[17]  Ulrike von Luxburg,et al.  Clustering Stability: An Overview , 2010, Found. Trends Mach. Learn..

[18]  Uwe Aickelin,et al.  Wavelet Feature Extraction and Genetic Algorithm for Biomarker Detection in Colorectal Cancer Data , 2013, Knowl. Based Syst..

[19]  Cynthia Rudin,et al.  An Interpretable Stroke Prediction Model using Rules and Bayesian Analysis , 2013, AAAI.

[20]  R. Kirschen,et al.  The Royal London Space Planning: an integration of space analysis and treatment planning: Part I: Assessing the space required to meet treatment objectives. , 2000, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[21]  H. Wickham,et al.  A Grammar of Data Manipulation , 2015 .

[22]  F. Herrera,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999 .

[23]  Toni Giorgino,et al.  Computing and Visualizing Dynamic Time Warping Alignments in R: The dtw Package , 2009 .

[24]  João Gama,et al.  Hierarchical Clustering of Time-Series Data Streams , 2008, IEEE Transactions on Knowledge and Data Engineering.

[25]  T. Palfrey,et al.  Anomalous Behavior in Public Goods Experiments: How Much and Why? , 1997 .

[26]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[27]  Thomas Seidl,et al.  Tracing Evolving Clusters by Subspace and Value Similarity , 2011, PAKDD.

[28]  J. Wolfers,et al.  Prediction Markets , 2003 .

[29]  Tony Jebara,et al.  Spectral Clustering and Embedding with Hidden Markov Models , 2007, ECML.

[30]  Li Wei,et al.  Semi-supervised time series classification , 2006, KDD '06.

[31]  Jian Pei,et al.  A brief survey on sequence classification , 2010, SKDD.

[32]  Michalis Vazirgiannis,et al.  Clustering validity assessment: finding the optimal partitioning of a data set , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[33]  E. Styvaktakis,et al.  Expert System for Classification and Analysis of Power System Events , 2002, IEEE Power Engineering Review.

[34]  M. Subha,et al.  Classification of Stock Index movement using k-Nearest Neighbours (k-NN) algorithm , 2012 .

[35]  Ramachandran Baskaran,et al.  A Survey on Internal Validity Measure for Cluster Validation , 2010 .

[36]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[37]  Shie Mannor,et al.  Concept Drift Detection Through Resampling , 2014, ICML.

[38]  Myra Spiliopoulou,et al.  MONIC: modeling and monitoring cluster transitions , 2006, KDD '06.

[39]  David J. Ketchen,et al.  THE APPLICATION OF CLUSTER ANALYSIS IN STRATEGIC MANAGEMENT RESEARCH: AN ANALYSIS AND CRITIQUE , 1996 .

[40]  T. Lumley,et al.  gplots: Various R Programming Tools for Plotting Data , 2015 .

[41]  Andreu Català,et al.  Rule-Based Learning Systems for Support Vector Machines , 2006, Neural Processing Letters.

[42]  András Kocsor,et al.  Application of a simple likelihood ratio approximant to protein sequence classification , 2006, Bioinform..

[43]  Peter Z. Revesz,et al.  Temporal Data Classification Using Linear Classifiers , 2009, ADBIS.

[44]  Sunil Prabhakar,et al.  A Rule-Based Classification Algorithm for Uncertain Data , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[45]  P. S. Sastry,et al.  A survey of temporal data mining , 2006 .

[46]  Hisao Ishibuchi,et al.  Adaptive fuzzy rule-based classification systems , 1996, IEEE Trans. Fuzzy Syst..

[47]  Thomas Seidl,et al.  Tracing Evolving Subspace Clusters in Temporal Climate Data , 2011, Data Mining and Knowledge Discovery.

[48]  E. Fama The Behavior of Stock-Market Prices , 1965 .

[49]  Carlos Agón,et al.  Time-series data mining , 2012, CSUR.

[50]  M. Friedman A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[51]  K. Hornik,et al.  Unbiased Recursive Partitioning: A Conditional Inference Framework , 2006 .

[52]  Ester Bernadó-Mansilla,et al.  Evolutionary rule-based systems for imbalanced data sets , 2008, Soft Comput..

[53]  Charu C. Aggarwal,et al.  On change diagnosis in evolving data streams , 2005, IEEE Transactions on Knowledge and Data Engineering.

[54]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[55]  Jianyong Wang,et al.  HARMONY: Efficiently Mining the Best Rules for Classification , 2005, SDM.

[56]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[57]  Mohammed J. Zaki Data Mining and Analysis: Fundamental Concepts and Algorithms , 2014 .

[58]  Ja-Ling Wu,et al.  Statistical analysis of space-varying morphological openings with flat structuring elements , 1996, IEEE Trans. Signal Process..

[59]  C. Mallows,et al.  A Method for Comparing Two Hierarchical Clusterings , 1983 .

[60]  Farid Melgani,et al.  Toward an Optimal SVM Classification System for Hyperspectral Remote Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[61]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[62]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[63]  J. Agrawal,et al.  State-of-the-Art in Stock Prediction Techniques , 2013 .

[64]  Hui Xiong,et al.  Understanding of Internal Clustering Validation Measures , 2010, 2010 IEEE International Conference on Data Mining.

[65]  Eamonn J. Keogh,et al.  On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration , 2002, Data Mining and Knowledge Discovery.

[66]  U. Fischbacher,et al.  Are People Conditionally Cooperative? Evidence from a Public Goods Experiment , 2001 .

[67]  Miin-Shen Yang,et al.  Alternative c-means clustering algorithms , 2002, Pattern Recognit..

[68]  Hui Zhang,et al.  A Non-parametric Wavelet Feature Extractor for Time Series Classification , 2004, PAKDD.

[69]  Hisao Ishibuchi,et al.  Voting in fuzzy rule-based systems for pattern classification problems , 1999, Fuzzy Sets Syst..

[70]  C. Faloutsos,et al.  cient Similarity Search In Sequence Databases , 2004 .

[71]  Cécile Amblard,et al.  Classification trees for time series , 2012, Pattern Recognit..

[72]  Vincent S. Tseng,et al.  Effective temporal data classification by integrating sequential pattern mining and probabilistic induction , 2009, Expert Syst. Appl..

[73]  Jan Ivar Larsen,et al.  Predicting Stock Prices Using Technical Analysis and Machine Learning , 2010 .

[74]  Ricardo J. G. B. Campello,et al.  Relative clustering validity criteria: A comparative overview , 2010 .

[75]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[76]  Michio Sugeno,et al.  A fuzzy-logic-based approach to qualitative modeling , 1993, IEEE Trans. Fuzzy Syst..

[77]  Michalis Vazirgiannis,et al.  Cluster validity methods: part I , 2002, SGMD.

[78]  M. Willinger,et al.  Weak Moral Motivation Leads to the Decline of Voluntary Contributions , 2011 .

[79]  Yanchang Zhao R and Data Mining: Examples and Case Studies , 2012 .

[80]  C. Keser,et al.  Conditional cooperation and voluntary contributions to public goods , 2000 .

[81]  Roberto Souto Maior de Barros,et al.  A comparative study on concept drift detectors , 2014, Expert Syst. Appl..

[82]  José A. Aguado,et al.  Rule-based classification of power quality disturbances using S-transform , 2012 .

[83]  Myra Spiliopoulou,et al.  Summarizing Cluster Evolution in Dynamic Environments , 2011, ICCSA.

[84]  Panos Kalnis,et al.  On Discovering Moving Clusters in Spatio-temporal Data , 2005, SSTD.

[85]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[86]  Urs Fischbacher,et al.  The behavioral validity of the strategy method in public good experiments , 2012 .

[87]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[88]  Myra Spiliopoulou,et al.  On exploiting the power of time in data mining , 2008, SKDD.

[89]  F. Hamprecht Introduction to Statistics , 2022 .

[90]  Alberto O. Mendelzon,et al.  Similarity-based queries for time series data , 1997, SIGMOD '97.

[91]  Myra Spiliopoulou,et al.  Tracing cluster transitions for different cluster types , 2009, Control. Cybern..

[92]  David B. Fogel,et al.  An Introduction to Evolutionary Computation , 2022 .

[93]  Dimitrios Gunopulos,et al.  Indexing multi-dimensional time-series with support for multiple distance measures , 2003, KDD '03.

[94]  Laura Firoiu,et al.  Clustering Time Series with Hidden Markov Models and Dynamic Time Warping , 1999 .

[95]  P. Rousseeuw,et al.  Partitioning Around Medoids (Program PAM) , 2008 .

[96]  Jiawei Han,et al.  CLARANS: A Method for Clustering Objects for Spatial Data Mining , 2002, IEEE Trans. Knowl. Data Eng..

[97]  Kurt Hornik,et al.  ctree : Conditional Inference Trees , 2015 .

[98]  Francesco Guala,et al.  Heterogeneous Agents in Public Goods Experiments , 2005 .

[99]  Jerry M. Mendel,et al.  Generating fuzzy rules by learning from examples , 1992, IEEE Trans. Syst. Man Cybern..

[100]  Wang Xy,et al.  Fuzzy clustering in the analysis of Fourier transform infrared spectra for cancer diagnosis. , 2006 .

[101]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[102]  G. W. Milligan,et al.  An examination of procedures for determining the number of clusters in a data set , 1985 .

[103]  A. Bifet,et al.  Early Drift Detection Method , 2005 .

[104]  Arnaud Giacometti,et al.  A Generic Framework for Rule-Based Classification , 2008 .

[105]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[106]  Xavier Robin,et al.  pROC: an open-source package for R and S+ to analyze and compare ROC curves , 2011, BMC Bioinformatics.

[107]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[108]  Alfred O. Hero,et al.  Adaptive evolutionary clustering , 2011, Data Mining and Knowledge Discovery.

[109]  Éric Gaussier,et al.  Generalized k-means-based clustering for temporal data under weighted and kernel time warp , 2016, Pattern Recognit. Lett..

[110]  Jacob K. Goeree,et al.  A theoretical analysis of altruism and decision error in public goods games , 1998 .

[111]  Michalis Vazirgiannis,et al.  Clustering validity checking methods: part II , 2002, SGMD.

[112]  M. Meilă Comparing clusterings---an information based distance , 2007 .

[113]  A. Lo,et al.  Stock Market Prices Do Not Follow Random Walks: Evidence from a Simple Specification Test , 1987 .

[114]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[115]  Marek Hlavac stargazer : LaTeX code for well-formatted regression and summary statistics tables (R package) , 2012 .

[116]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[117]  I. Kaul Global Public Goods , 2016 .

[118]  Olfa Nasraoui,et al.  Stream-dashboard: a framework for mining, tracking and validating clusters in a data stream , 2012, BigMine '12.

[119]  Ananish Chaudhuri Sustaining cooperation in laboratory public goods experiments: a selective survey of the literature , 2011 .

[120]  Ananish Chaudhuri,et al.  Conditional cooperation and voluntary contributions to a public good , 2006 .

[121]  Olatz Arbelaitz,et al.  An extensive comparative study of cluster validity indices , 2013, Pattern Recognit..

[122]  Jae C. Oh,et al.  Improving Learning of Genetic Rule-Based Classifier Systems , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[123]  Michael Kosfeld,et al.  Conditional Cooperation and Costly Monitoring Explain Success in Forest Commons Management , 2010, Science.

[124]  Patrick Henry Winston,et al.  Artificial intelligence (2nd ed.) , 1984 .

[125]  Marina Meila,et al.  Comparing Clusterings by the Variation of Information , 2003, COLT.

[126]  Roman Garnett,et al.  Learning from data streams with concept drift , 2008 .

[127]  David Ardia,et al.  Differential Evolution (DEoptim) for Non-Convex Portfolio Optimization , 2010 .

[128]  Martin Dufwenberg,et al.  Author's Personal Copy Games and Economic Behavior the Framing of Games and the Psychology of Play , 2022 .

[129]  H. Ishibuchi,et al.  Distributed representation of fuzzy rules and its application to pattern classification , 1992 .

[130]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[131]  Michalis Vazirgiannis,et al.  Quality Scheme Assessment in the Clustering Process , 2000, PKDD.

[132]  Urs Fischbacher,et al.  Learning and Peer Effects , 2008 .

[133]  Wolfgang J. Luhan,et al.  Cedex Discussion Paper Series , 2022 .

[134]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[135]  David W. Aha,et al.  A Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms , 1997, Artificial Intelligence Review.

[136]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[137]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[138]  Ada Wai-Chee Fu,et al.  Efficient time series matching by wavelets , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[139]  Jiawei Han,et al.  Data Mining: Concepts and Techniques, Second Edition , 2006, The Morgan Kaufmann series in data management systems.

[140]  Guoyin Wang,et al.  An automatic method to determine the number of clusters using decision-theoretic rough set , 2014, Int. J. Approx. Reason..

[141]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[142]  Eréndira Rendón,et al.  Internal versus External cluster validation indexes , 2011 .

[143]  Pasi Fränti,et al.  Set Matching Measures for External Cluster Validity , 2016, IEEE Transactions on Knowledge and Data Engineering.

[144]  Tea Tusar,et al.  Differential Evolution versus Genetic Algorithms in Multiobjective Optimization , 2007, EMO.