A Generalised Signature Method for Time Series

The `signature method' refers to a collection of feature extraction techniques for multimodal sequential data, derived from the theory of controlled differential equations. Variations exist as many authors have proposed modifications to the method, so as to improve some aspect of it. Here, we introduce a \emph{generalised signature method} that contains these variations as special cases, and groups them conceptually into \emph{augmentations}, \emph{windows}, \emph{transforms}, and \emph{rescalings}. Within this framework we are then able to propose novel variations, and demonstrate how previously distinct options may be combined. We go on to perform an extensive empirical study on 26 datasets as to which aspects of this framework typically produce the best results. Combining the top choices produces a canonical pipeline for the generalised signature method, which demonstrates state-of-the-art accuracy on benchmark problems in multivariate time series classification.

[1]  Csaba Toth,et al.  Bayesian Learning from Sequential Data using Gaussian Processes with Signature Covariances , 2020, ICML.

[2]  Benjamin M. Marlin,et al.  A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification , 2016, NIPS.

[3]  Xin Zhang,et al.  LPSNet: A Novel Log Path Signature Feature Based Hand Gesture Recognition Framework , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[4]  Terry Lyons,et al.  Learning from the past, predicting the statistics for the future, learning an evolving system , 2013, 1309.0260.

[5]  Davide Anguita,et al.  Transition-Aware Human Activity Recognition Using Smartphones , 2016, Neurocomputing.

[6]  Lianwen Jin,et al.  DeepWriterID: An End-to-End Online Text-Independent Writer Identification System , 2015, IEEE Intelligent Systems.

[7]  Terry Lyons,et al.  Learning stochastic differential equations using RNN with log signature features , 2019, ArXiv.

[8]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[9]  Yu Wu,et al.  Signature features with the visibility transformation , 2020, ArXiv.

[10]  David Duvenaud,et al.  Latent Ordinary Differential Equations for Irregularly-Sampled Time Series , 2019, NeurIPS.

[11]  Andrey Kormilitzin,et al.  A Primer on the Signature Method in Machine Learning , 2016, ArXiv.

[12]  Eamonn J. Keogh,et al.  The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances , 2016, Data Mining and Knowledge Discovery.

[13]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[14]  Andrey Kormilitzin,et al.  Utilization of the Signature Method to Identify the Early Onset of Sepsis From Multivariate Physiological Time Series in Critical Care Monitoring , 2020, Critical care medicine.

[15]  Ole Tange,et al.  GNU Parallel: The Command-Line Power Tool , 2011, login Usenix Mag..

[16]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[17]  Terry Lyons,et al.  A signature-based machine learning model for distinguishing bipolar disorder and borderline personality disorder , 2017, Translational Psychiatry.

[18]  K. E. A. Saunders,et al.  Application of the Signature Method to Pattern Recognition in the CEQUEL Clinical Trial , 2016, 1606.02074.

[19]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Harald Oberhauser,et al.  Variational Gaussian Processes with Signature Covariances , 2019, ArXiv.

[21]  Lianwen Jin,et al.  Rotation-free online handwritten character recognition using dyadic path signature features, hanging normalization, and deep neural network , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[22]  Benjamin Graham,et al.  The iisignature library: efficient calculation of iterated-integral signatures and log signatures , 2017, ACM Trans. Math. Softw..

[23]  Eamonn J. Keogh,et al.  Time series shapelets: a new primitive for data mining , 2009, KDD.

[24]  Germain Forestier,et al.  Deep learning for time series classification: a review , 2018, Data Mining and Knowledge Discovery.

[25]  Pete Warden,et al.  Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition , 2018, ArXiv.

[26]  Peter K. Friz,et al.  Multidimensional Stochastic Processes as Rough Paths: Theory and Applications , 2010 .

[27]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[28]  Francesca Mangili,et al.  Should We Really Use Post-Hoc Tests Based on Mean-Ranks? , 2015, J. Mach. Learn. Res..

[29]  James Morrill,et al.  The Signature-Based Model for Early Detection of Sepsis From Electronic Health Records in the Intensive Care Unit , 2019, 2019 Computing in Cardiology (CinC).

[30]  Adeline Fermanian Embedding and learning with signatures , 2019, ArXiv.

[31]  École d'été de probabilités de Saint-Flour,et al.  Differential equations driven by rough paths , 2007 .

[32]  Michael Flynn,et al.  The UEA multivariate time series classification archive, 2018 , 2018, ArXiv.

[33]  Patrick Kidger,et al.  Generalised Interpretable Shapelets for Irregular Time Series , 2020, ArXiv.

[34]  Terry Lyons,et al.  A feature set for streams and an application to high-frequency financial tick data , 2014, BigDataScience '14.

[35]  Katherine A. Heller,et al.  Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier , 2017, ICML.

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[37]  Terry Lyons,et al.  Discretely sampled signals and the rough Hoff process , 2013, 1310.4054.

[38]  Lianwen Jin,et al.  Online Signature Verification Using Recurrent Neural Network and Length-Normalized Path Signature Descriptor , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[39]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[40]  J. Schmidhuber,et al.  The Sacred Infrastructure for Computational Research , 2017, SciPy.

[41]  Jeremy Francis Reizenstein Iterated-integral signatures in machine learning , 2019 .

[42]  Terry Lyons,et al.  Sketching the order of events , 2017, ArXiv.

[43]  Patrick Kidger,et al.  Neural Controlled Differential Equations for Irregular Time Series , 2020, NeurIPS.

[44]  Cordelia Schmid,et al.  Leveraging the Path Signature for Skeleton-based Human Action Recognition , 2017, ArXiv.

[45]  Patrick Kidger,et al.  Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU , 2020, ArXiv.

[46]  Franz J. Király,et al.  Kernels for sequentially ordered data , 2016, J. Mach. Learn. Res..

[47]  Lars Schmidt-Thieme,et al.  Learning time-series shapelets , 2014, KDD.