论文信息 - Classification of Multivariate Time Series and Structured Data Using Constructive Induction

2016 - Diabetes research and clinical practice

Assessing occurrence of hypoglycemia and its severity from electronic health records of patients with type 2 diabetes mellitus.

AIMS Accurate measures of hypoglycemia within electronic health records (EHR) can facilitate clinical population management and research. We quantify the occurrence of serious and mild-to-moderate hypoglycemia in a large EHR database in the US, comparing estimates based only on structured data to those from structured data and natural language processing (NLP) of clinical notes. METHODS This cohort study included patients with type 2 diabetes identified from January 2009 through March 2014. We compared estimates of occurrence of hypoglycemia derived from diagnostic codes to those recorded within clinical notes and classified via NLP. Measures of hypoglycemia from only structured data (ICD-9 Algorithm), only note mentions (NLP Algorithm), and either structured data or notes (Combined Algorithm) were compared with estimates of the period prevalence, incidence rate, and event rate of hypoglycemia, overall and by seriousness. RESULTS Of the 844,683 eligible patients, 119,695 had at least one recorded hypoglycemic event identified with ICD-9 or NLP. The period prevalence of hypoglycemia was 12.4%, 25.1%, and 32.2% for the ICD-9 Algorithm, NLP Algorithm, and Combined Algorithm, respectively. There were 6128 apparent non-serious events utilizing the ICD-9 Algorithm, which increased to 152,987 non-serious events within the Combined Algorithm. CONCLUSIONS Ascertainment of events from clinical notes more than doubled the completeness of hypoglycemia capture overall relative to measures from structured data, and increased capture of non-serious events more than 20-fold. The structured data and clinical notes are complementary within the EHR, and both need to be considered in order to fully assess the occurrence of hypoglycemia.

2011 - Physical review. E, Statistical, nonlinear, and soft matter physics

Estimation of Kramers-Moyal coefficients at low sampling rates.

An optimization procedure for the estimation of Kramers-Moyal coefficients from stationary, one-dimensional, Markovian time series data is presented. The method takes advantage of a recently reported approach that allows one to calculate exact finite sampling interval effects by solving the adjoint Fokker-Planck equation. Therefore, it is well suited for the analysis of sparsely sampled time series. The optimization can be performed either by making a parametric ansatz for drift and diffusion functions or parameter free. We demonstrate the power of the method in several numerical examples with synthetic time series.

2019 - IEEE Transactions on Circuits and Systems I: Regular Papers

SensorNet: A Scalable and Low-Power Deep Convolutional Neural Network for Multimodal Data Classification

This paper presents SensorNet which is a scalable and low-power embedded deep convolutional neural network (DCNN), designed to classify multimodal time series signals. Time series signals generated by different sensor modalities with different sampling rates are first converted to images (2-D signals), and then DCNN is utilized to automatically learn shared features in the images and perform the classification. SensorNet: 1) is scalable as it can process different types of time series data with variety of input channels and sampling rates; 2) does not need to employ separate signal processing techniques for processing the data generated by each sensor modality; 3) does not require expert knowledge for extracting features for each sensor data; 4) makes it easy and fast to adapt to new sensor modalities with a different sampling rate; 5) achieves very high detection accuracy for different case studies; and 6) has a very efficient architecture which makes it suitable to be deployed at Internet of Things and wearable devices. A custom low-power hardware architecture is also designed for the efficient deployment of SensorNet at embedded real-time systems. SensorNet performance is evaluated using three different case studies including physical activity monitoring, stand-alone tongue drive system, and stress detection, and it achieves an average detection accuracy of 98%, 96.2%, and 94% for each case study, respectively. We implement SensorNet using our custom hardware architecture on Xilinx FPGA (Artix-7) which on average consumes 246-<inline-formula> <tex-math notation="LaTeX">$\mu \text{J}$ </tex-math></inline-formula> energy. To further reduce the power consumption, SensorNet is implemented using application-specified integrated circuit at the post layout level in 65-nm CMOS technology which consumes approximately <inline-formula> <tex-math notation="LaTeX">$8\times $ </tex-math></inline-formula> lower power compared to the FPGA implementation. In addition, SensorNet is implemented on NVIDIA Jetson TX2 SoC (CPU + GPU) and compared to TX2 single-core CPU and GPU implementations, FPGA-based SensorNet obtains <inline-formula> <tex-math notation="LaTeX">$15\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula> improvement in energy consumption.

1995 - Technological Forecasting and Social Change

Working Less and Living Longer: Long-term Trends in Working Time and Time Budgets

Abstract Analyses of time series data beginning in the mid-nineteenth century in the industrialized nations, especially the United Kingdom, show that on average people are working significantly less while living longer. Although the average career length has remained around 40 years, the total life hours worked shrank for an average British worker from 124,000 hours in 1856 to 69,000 in 1981. The fraction of disposable lifetime hours spent working declined from 50% to 20%. Meanwhile the female share of career years doubled to 30%. If the long-term trends continue at their historic rates, the work week might average 27 hours by the year 2050. The secular trend away from the formalized work contract to other socially obligatory activities and free time implies numerous challenges for human societies.

2016 - Expert Syst. Appl.

Fuzzy time series forecasting method based on hesitant fuzzy sets

We propose a fuzzy time series forecasting method using hesitant fuzzy sets.An aggregation operator is also proposed to aggregate hesitant fuzzy elements.Hesitancy is introduced by using multiple fuzzification methods.University of Alabama enrollments and SBI share prices are forecasted.Performance of method is measured in terms of MSE and AFER. This study proposes a fuzzy time series forecasting method based on hesitant fuzzy sets for forecasting in the environment of hesitant information. The proposed method addresses the problem of establishing a common membership grade for the situation when multiple fuzzification methods are available to fuzzify time series data. An aggregation operator for aggregating hesitant information is also proposed in the study. The proposed method is implemented to forecast enrollment at University of Alabama and price of state bank of India (SBI) share at Bombay stock exchange (BSE), India. In both time series data are fuzzified with triangular fuzzy sets constructed using intervals of equal and unequal length. The performance of the proposed method in forecasting student enrollments and SBI share price is measured in terms of root mean square and average forecasting errors. Statistical validation and performance analysis is also carried out to validate the proposed forecasting method.

2007

Connectivity Inference between Neural Structures via Partial Directed Coherence

Abstract This paper describes the rigorous asymptotic distributions of the recently introduced partial directed coherence (PDC) – a frequency domain description of Granger causality between multivariate time series represented by vector autoregressive models. We show that, when not zero, PDC is asymptotically normally distributed and therefore provides means of comparing different strengths of connection between observed time series. Zero PDC indicates an absence of a direct connection between time series, and its otherwise asymptotically normal behavior degenerates into that of a mixture of variables allowing the computation of rigorous thresholds for connectivity tests using either numerical integration or approximate numerical methods. A Monte Carlo study illustrates the power of the test under PDC nullity. An analysis of electroencephalographic data, before and during an epileptic seizure episode, is used to portray the usefulness of the test in a real application.

2012 - 2012 IEEE 12th International Conference on Data Mining

Clustering Time Series Using Unsupervised-Shapelets

Time series clustering has become an increasingly important research topic over the past decade. Most existing methods for time series clustering rely on distances calculated from the entire raw data using the Euclidean distance or Dynamic Time Warping distance as the distance measure. However, the presence of significant noise, dropouts, or extraneous data can greatly limit the accuracy of clustering in this domain. Moreover, for most real world problems, we cannot expect objects from the same class to be equal in length. As a consequence, most work on time series clustering only considers the clustering of individual time series "behaviors," e.g., individual heart beats or individual gait cycles, and contrives the time series in some way to make them all equal in length. However, contriving the data in such a way is often a harder problem than the clustering itself. In this work, we show that by using only some local patterns and deliberately ignoring the rest of the data, we can mitigate the above problems and cluster time series of different lengths, i.e., cluster one heartbeat with multiple heartbeats. To achieve this we exploit and extend a recently introduced concept in time series data mining called shapelets. Unlike existing work, our work demonstrates for the first time the unintuitive fact that shapelets can be learned from unlabeled time series. We show, with extensive empirical evaluation in diverse domains, that our method is more accurate than existing methods. Moreover, in addition to accurate clustering results, we show that our work also has the potential to give insights into the domains to which it is applied.

2009 - Artificial intelligence in medicine

Matching incomplete time series with dynamic time warping: an algorithm and an application to post-stroke rehabilitation

OBJECTIVE The purpose of this study was to assess the performance of a real-time ("open-end") version of the dynamic time warping (DTW) algorithm for the recognition of motor exercises. Given a possibly incomplete input stream of data and a reference time series, the open-end DTW algorithm computes both the size of the prefix of reference which is best matched by the input, and the dissimilarity between the matched portions. The algorithm was used to provide real-time feedback to neurological patients undergoing motor rehabilitation. METHODS AND MATERIALS We acquired a dataset of multivariate time series from a sensorized long-sleeve shirt which contains 29 strain sensors distributed on the upper limb. Seven typical rehabilitation exercises were recorded in several variations, both correctly and incorrectly executed, and at various speeds, totaling a data set of 840 time series. Nearest-neighbour classifiers were built according to the outputs of open-end DTW alignments and their global counterparts on exercise pairs. The classifiers were also tested on well-known public datasets from heterogeneous domains. RESULTS Nonparametric tests show that (1) on full time series the two algorithms achieve the same classification accuracy (p-value =0.32); (2) on partial time series, classifiers based on open-end DTW have a far higher accuracy (kappa=0.898 versus kappa=0.447;p<10(-5)); and (3) the prediction of the matched fraction follows closely the ground truth (root mean square <10%). The results hold for the motor rehabilitation and the other datasets tested, as well. CONCLUSIONS The open-end variant of the DTW algorithm is suitable for the classification of truncated quantitative time series, even in the presence of noise. Early recognition and accurate class prediction can be achieved, provided that enough variance is available over the time span of the reference. Therefore, the proposed technique expands the use of DTW to a wider range of applications, such as real-time biofeedback systems.

2012 - BMC Systems Biology

Estimation of dynamic flux profiles from metabolic time series data

BackgroundAdvances in modern high-throughput techniques of molecular biology have enabled top-down approaches for the estimation of parameter values in metabolic systems, based on time series data. Special among them is the recent method of dynamic flux estimation (DFE), which uses such data not only for parameter estimation but also for the identification of functional forms of the processes governing a metabolic system. DFE furthermore provides diagnostic tools for the evaluation of model validity and of the quality of a model fit beyond residual errors. Unfortunately, DFE works only when the data are more or less complete and the system contains as many independent fluxes as metabolites. These drawbacks may be ameliorated with other types of estimation and information. However, such supplementations incur their own limitations. In particular, assumptions must be made regarding the functional forms of some processes and detailed kinetic information must be available, in addition to the time series data.ResultsThe authors propose here a systematic approach that supplements DFE and overcomes some of its shortcomings. Like DFE, the approach is model-free and requires only minimal assumptions. If sufficient time series data are available, the approach allows the determination of a subset of fluxes that enables the subsequent applicability of DFE to the rest of the flux system. The authors demonstrate the procedure with three artificial pathway systems exhibiting distinct characteristics and with actual data of the trehalose pathway in Saccharomyces cerevisiae.ConclusionsThe results demonstrate that the proposed method successfully complements DFE under various situations and without a priori assumptions regarding the model representation. The proposed method also permits an examination of whether at all, to what degree, or within what range the available time series data can be validly represented in a particular functional format of a flux within a pathway system. Based on these results, further experiments may be designed to generate data points that genuinely add new information to the structure identification and parameter estimation tasks at hand.

1993

The Econometrics of Panel Data

This completely restructured, updated third edition of The Econometrics of Panel Data, first published in 1992, provides a general overview of the econometrics of panel data, both from a theoretical and from an applied viewpoint. Since the pioneering papers by Kuh, Mundlak, Hoch and Balestra and Nerlove, the pooling of cross section and time series data has become an increasingly popular way of quantifying economic relationships. Each series provides information lacking in the other, so a combination of both leads to more accurate and reliable results than would be achievable by one type of series alone. This third, enhanced edition provides a complete and up to date presentation of theoretical developments as well as surveys about how econometric tools are used to study firms and household's behaviors. It contains eleven entirely new chapters while the others have been largely revised to account for recent developments in the field.

2006 - Comput. Stat. Data Anal.

A periodogram-based metric for time series classification

The statistical discrimination and clustering literature has studied the problem of identifying similarities in time series data. Some studies use non-parametric approaches for splitting a set of time series into clusters by looking at their Euclidean distances in the space of points. A new measure of distance between time series based on the normalized periodogram is proposed. Simulation results comparing this measure with others parametric and non-parametric metrics are provided. In particular, the classification of time series as stationary or as non-stationary is discussed. The use of both hierarchical and non-hierarchical clustering algorithms is considered. An illustrative example with economic time series data is also presented.

2019 - IEEE Transactions on Knowledge and Data Engineering

Real-Time Change Point Detection with Application to Smart Home Time Series Data

Change Point Detection (CPD) is the problem of discovering time points at which the behavior of a time series changes abruptly. In this paper, we present a novel real-time nonparametric change point detection algorithm called SEP, which uses Separation distance as a divergence measure to detect change points in high-dimensional time series. Through experiments on artificial and real-world datasets, we demonstrate the usefulness of the proposed method in comparison with existing methods.

2011 - Knowledge and Information Systems

Improving SVM classification on imbalanced time series data sets with ghost points

Imbalanced data sets present a particular challenge to the data mining community. Often, it is the rare event that is of interest and the cost of misclassifying the rare event is higher than misclassifying the usual event. When the data is highly skewed toward the usual, it can be very difficult for a learning system to accurately detect the rare event. There have been many approaches in recent years for handling imbalanced data sets, from under-sampling the majority class to adding synthetic points to the minority class in feature space. However, distances between time series are known to be non-Euclidean and non-metric, since comparing time series requires warping in time. This fact makes it impossible to apply standard methods like SMOTE to insert synthetic data points in feature spaces. We present an innovative approach that augments the minority class by adding synthetic points in distance spaces. We then use Support Vector Machines for classification. Our experimental results on standard time series show that our synthetic points significantly improve the classification rate of the rare events, and in most cases also improves the overall accuracy of SVMs. We also show how adding our synthetic points can aid in the visualization of time series data sets.

2007 - SIAM J. Appl. Dyn. Syst.

Application of the 0-1 Test for Chaos to Experimental Data

A reliable and efficient method of distinguishing between chaotic and nonchaotic behavior in noise-contaminated, but essentially stationary and deterministic, time series data has far reaching applications. Recently, we proposed a new method of detecting chaos which applies directly to the time series data and does not require phase space reconstruction. To illustrate the effectiveness of the method for experimental data, we analyze data from a bipolar motor.

2015 - Machine Learning

Using causal discovery for feature selection in multivariate numerical time series

Time series data contains temporal ordering, which makes its feature selection different from the normal feature selection. Feature selection in multivariate time series has two tasks: identifying the relevant features and finding their effective window sizes of lagged values. The methods extended from normal feature selection methods do not solve this two-dimensional feature selection problem since they do not take lagged observations of features into consideration. In this paper, we present a method using the Granger causality discovery to identify causal features with effective sliding window sizes in multivariate numerical time series. The proposed method considers the influence of lagged observations of features on the target time series. We compare our proposed feature selection method with several normal feature selection methods on multivariate time series data using three well-known modeling methods. Our method outperforms other methods for predicting future values of target time series. In a real world case study on water quality monitoring data, we show that the features selected by our method contain four out of five features used by domain experts, and prediction performance on our features is better than that on features of domain experts using three modeling methods.

2018 - IEEE Transactions on Signal Processing

Forecasting Time Series With VARMA Recursions on Graphs

Graph-based techniques emerged as a choice to deal with the dimensionality issues in modeling multivariate time series. However, there is yet no complete understanding of how the underlying structure could be exploited to ease this task. This paper provides contributions in this direction by considering the forecasting of a process evolving over a graph. We make use of the (approximate) time-vertex stationarity assumption, i.e., time-varying graph signals whose first- and second-order statistical moments are invariant over time and correlated to a known graph topology. The latter is combined with vector autoregressive and vector autoregressive moving average models to tackle the dimensionality issues present in predicting the temporal evolution of multivariate time series. We find out that by projecting the data to the graph spectral domain the multivariate model estimation reduces to that of fitting a number of uncorrelated univariate autoregressive–moving-average models and an optimal low-rank data representation can be exploited so as to further reduce the estimation costs. In the case that the multivariate process can be observed at a subset of nodes, the proposed models extend naturally to Kalman filtering on graphs allowing for optimal tracking. Numerical experiments with both synthetic and real data validate the proposed approach and highlight its benefits over state-of-the-art alternatives.

2019

Nonlinear Time Series: Semiparametric and Nonparametric Methods

Useful in the theoretical and empirical analysis of nonlinear time series data, semiparametric methods have received extensive attention in the economics and statistics communities over the past twenty years. Recent studies show that semiparametric methods and models may be applied to solve dimensionality reduction problems arising from using fully nonparametric models and methods. Answering the call for an up-to-date overview of the latest developments in the field, "Nonlinear Time Series: Semiparametric and Nonparametric Methods" focuses on various semiparametric methods in model estimation, specification testing, and selection of time series data.After a brief introduction, this book examines semiparametric estimation and specification methods and then applies these approaches to a class of nonlinear continuous-time models with real-world data. It also assesses some newly proposed semiparametric estimation procedures for time series data with long-range dependence. Even though this book only deals with climatological and financial data, the estimation and specifications methods discussed can be applied to models with real-world data in many disciplines. This resource covers key methods in time series analysis and provides the necessary theoretical details. The latest applied finance and financial econometrics results and applications presented in this book enable researchers and graduate students to keep abreast of developments in the field.

2009

Anomaly detection for symbolic sequences and time series data

This thesis deals with the problem of anomaly detection for sequence data. Anomaly detection has been a widely researched problem in several application domains such as system health management, intrusion detection, health-care, bio-informatics, fraud detection, and mechanical fault detection. Traditional anomaly detection techniques analyze each data instance (as a univariate or multivariate record) independently, and ignore the sequential aspect of the data. Often, anomalies in sequences can be detected only by analyzing data instances together as a sequence, and hence cannot detected by traditional anomaly detection techniques. The problem of anomaly detection for sequence data is a rich area of research because of two main reasons. First, sequences can be of different types, e.g., symbolic sequences, time series data, etc., and each type of sequence poses unique set of problems. Second, anomalies in sequences can be defined in multiple ways and hence there are different problem formulations. In this thesis we focus on solving one particular problem formulation called semi-supervised anomaly detection. We study the problem separately for symbolic sequences, univariate time series data, and multivariate time series data. The state of art on anomaly detection for sequences is limited and fragmented across application domains. For symbolic sequences, several techniques have been proposed within specific domains, but it is not well-understood as to how a technique developed for one domain would perform in a completely different domain. For univariate time series data, limited techniques exist, and are only evaluated for specific domains, while for multivariate time series data, anomaly detection research is relatively untouched. This thesis has two key goals. First goal is to develop novel anomaly detection techniques for different types of sequences which perform better than existing techniques across a variety of application domains. The second goal is to identify the best anomaly detection technique for a given application domain. By realizing the first goal, we develop a suite of anomaly detection techniques for a domain scientist to choose from, while the second goal will help the scientist to choose the technique best suited for the task. To achieve the first goal, we develop several novel anomaly detection techniques for univariate symbolic sequences, univariate time series data, and multivariate time series data. We provide extensive experimental evaluation of the proposed techniques on data sets collected across diverse domains and generated from data generators, also developed as part of this thesis. We show how the proposed techniques can be used to detect anomalies which translate to critical events in domains such as aircraft safety, intrusion detection, and patient health management. The techniques proposed in this thesis are shown to outperform existing techniques on many data sets. The technique proposed for multivariate time series data is one of the very first anomaly detection technique that can detect complex anomalies in such data. To achieve the second goal, we study the relationship between anomaly detection techniques and the nature of the data on which they are applied. A novel analysis framework, Reference Based Analysis (RBA), is proposed that can map a given data set (of any type) into a multivariate continuous space with respect to a reference data set. We apply the RBA framework to not only visualize and understand complex data types, such as multivariate categorical data and symbolic sequence data, but also to extract data driven features from symbolic sequences, which when used with traditional anomaly detection techniques are shown to consistently outperform the state of art anomaly detection techniques for these complex data types. Two novel techniques for symbolic sequences, WIN1D and WIN 2D are proposed using the RBA framework which perform better than the best technique for each different data set.

2019 - Electronics

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

Multivariable time series prediction has been widely studied in power energy, aerology, meteorology, finance, transportation, etc. Traditional modeling methods have complex patterns and are inefficient to capture long-term multivariate dependencies of data for desired forecasting accuracy. To address such concerns, various deep learning models based on Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) methods are proposed. To improve the prediction accuracy and minimize the multivariate time series data dependence for aperiodic data, in this article, Beijing PM2.5 and ISO-NE Dataset are analyzed by a novel Multivariate Temporal Convolution Network (M-TCN) model. In this model, multi-variable time series prediction is constructed as a sequence-to-sequence scenario for non-periodic datasets. The multichannel residual blocks in parallel with asymmetric structure based on deep convolution neural network is proposed. The results are compared with rich competitive algorithms of long short term memory (LSTM), convolutional LSTM (ConvLSTM), Temporal Convolution Network (TCN) and Multivariate Attention LSTM-FCN (MALSTM-FCN), which indicate significant improvement of prediction accuracy, robust and generalization of our model.

2002 - Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining

A unifying framework for detecting outliers and change points from non-stationary time series data

We are concerned with the issues of outlier detection and change point detection from a data stream. In the area of data mining, there have been increased interest in these issues since the former is related to fraud detection, rare event discovery, etc., while the latter is related to event/trend by change detection, activity monitoring, etc. Specifically, it is important to consider the situation where the data source is non-stationary, since the nature of data source may change over time in real applications. Although in most previous work outlier detection and change point detection have not been related explicitly, this paper presents a unifying framework for dealing with both of them on the basis of the theory of on-line learning of non-stationary time series. In this framework a probabilistic model of the data source is incrementally learned using an on-line discounting learning algorithm, which can track the changing data source adaptively by forgetting the effect of past data gradually. Then the score for any given data is calculated to measure its deviation from the learned model, with a higher score indicating a high possibility of being an outlier. Further change points in a data stream are detected by applying this scoring method into a time series of moving averaged losses for prediction using the learned model. Specifically we develop an efficient algorithms for on-line discounting learning of auto-regression models from time series data, and demonstrate the validity of our framework through simulation and experimental applications to stock market data analysis.

Classification of Multivariate Time Series and Structured Data Using Constructive Induction

Efficient forward propagation of time-sequences in convolutional neural networks using Deep Shifting

Machine Learning for Biomedical Time Series Classification: From Shapelets to Deep Learning

Automatic Recognition of the Unconscious Reactions from Physiological Signals

Feature Extraction over Multiple Representations for Time Series Classification

Trajectory Analysis on Spherical Self-Organizing Maps with Application to Gesture Recognition

A multi-valued and sequential-labeled decision tree method for recommending sequential patterns in cold-start situations

Adaptive framework for multivariate stream data processing in data-centric sensor applications

Multivariate Stream Data Classification Using Simple Text Classifiers

Multivariate time series classification with parametric derivative dynamic time warping

Traces of Unconscious Mental Processes in Introspective Reports and Physiological Responses

Feature Based Classification of Astronomical Transients

Convolution Kernels for Discriminative Learning from Streaming Text

A review on time series data mining

Combining discrete SVM and fixed cardinality warping distances for multivariate time series classification

Fast Randomized Model Generation for Shapelet-Based Time Series Classification

Dynamic Data Analysis and Data Mining for Prediction of Clinical Stability

Prediction of Clinical Conditions after Coronary Bypass Surgery using Dynamic Data Analysis

Stacking for multivariate time series classification

Early Classification of Ongoing Observation

Multilinear function factorisation for time series feature extraction