Embedded Dimension and Time Series Length. Practical Influence on Permutation Entropy and Its Applications

Permutation Entropy (PE) is a time series complexity measure commonly used in a variety of contexts, with medicine being the prime example. In its general form, it requires three input parameters for its calculation: time series length N, embedded dimension m, and embedded delay τ. Inappropriate choices of these parameters may potentially lead to incorrect interpretations. However, there are no specific guidelines for an optimal selection of N, m, or τ, only general recommendations such as N>>m!, τ=1, or m=3,…,7. This paper deals specifically with the study of the practical implications of N>>m!, since long time series are often not available, or non-stationary, and other preliminary results suggest that low N values do not necessarily invalidate PE usefulness. Our study analyses the PE variation as a function of the series length N and embedded dimension m in the context of a diverse experimental set, both synthetic (random, spikes, or logistic model time series) and real–world (climatology, seismic, financial, or biomedical time series), and the classification performance achieved with varying N and m. The results seem to indicate that shorter lengths than those suggested by N>>m! are sufficient for a stable PE calculation, and even very short time series can be robustly classified based on PE measurements before the stability point is reached. This may be due to the fact that there are forbidden patterns in chaotic time series, not all the patterns are equally informative, and differences among classes are already apparent at very short lengths.

[1]  Marimuthu Palaniswami,et al.  Stability, Consistency and Performance of Distribution Entropy in Analysing Short Length Heart Rate Variability (HRV) Signal , 2017, Front. Physiol..

[2]  Rongxi Zhou,et al.  Applications of Entropy in Finance: A Review , 2013, Entropy.

[3]  Karsten Keller,et al.  Ordinal Patterns, Entropy, and EEG , 2014, Entropy.

[4]  Nathaniel H. Hunt,et al.  The Appropriate Use of Approximate Entropy and Sample Entropy with Short Data Sets , 2012, Annals of Biomedical Engineering.

[5]  P. Jones,et al.  Multi-Scale Entropy Analysis as a Method for Time-Series Analysis of Climate Data , 2015 .

[6]  Marcelo Risk,et al.  Classification of Normal and Pre-Ictal EEG Signals Using Permutation Entropies and a Generalized Linear Model as a Classifier , 2017, Entropy.

[7]  Marko Robnik-Sikonja,et al.  Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF , 2004, Applied Intelligence.

[8]  Andreas Holzinger,et al.  Selection of entropy-measure parameters for knowledge discovery in heart rate variability data , 2014, BMC Bioinformatics.

[9]  K Lehnertz,et al.  Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: dependence on recording region and brain state. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  James McNames,et al.  An automatic beat detection algorithm for pressure signals , 2005, IEEE Transactions on Biomedical Engineering.

[11]  Yaxing Wei,et al.  Daymet: Daily Surface Weather Data on a 1-km Grid for North America, Version 2 , 2014 .

[12]  David Cuesta-Frau,et al.  Noisy EEG signals classification based on entropy metrics. Performance assessment using first and second generation statistics , 2017, Comput. Biol. Medicine.

[13]  Abdulhamit Subasi,et al.  EEG signal classification using wavelet feature extraction and a mixture of expert model , 2007, Expert Syst. Appl..

[14]  C. Peng,et al.  A Strategy to Reduce Bias of Entropy Estimates in Resting-State fMRI Signals , 2018, Front. Neurosci..

[15]  Chen Chen,et al.  Classification of single-channel EEG signals for epileptic seizures detection based on hybrid features , 2018, Technology and health care : official journal of the European Society for Engineering and Medicine.

[16]  Massimiliano Zanin,et al.  Forbidden patterns in financial time series. , 2007, Chaos.

[17]  Luciano Zunino,et al.  Forbidden patterns, permutation entropy and stock market inefficiency , 2009 .

[18]  R Alcaraz,et al.  Study of Sample Entropy ideal computational parameters in the estimation of atrial fibrillation organization from the ECG , 2010, 2010 Computing in Cardiology.

[19]  O A Rosso,et al.  Distinguishing noise from chaos. , 2007, Physical review letters.

[20]  John B. Moore,et al.  High Performance Control , 1997 .

[21]  Alejandra Figliola,et al.  Entropy analysis of the dynamics of El Niño/Southern Oscillation during the Holocene , 2010 .

[22]  David Cuesta-Frau,et al.  A new algorithm for quadratic sample entropy optimization for very short biomedical signals: Application to blood pressure records , 2014, Comput. Methods Programs Biomed..

[23]  A. Plastino,et al.  Permutation entropy of fractional Brownian motion and fractional Gaussian noise , 2008 .

[24]  Germán Castellanos-Domínguez,et al.  Unsupervised feature relevance analysis applied to improve ECG heartbeat clustering , 2012, Comput. Methods Programs Biomed..

[25]  Yali Zhang,et al.  Permutation entropy analysis of financial time series based on Hill's diversity number , 2017, Commun. Nonlinear Sci. Numer. Simul..

[26]  David Cuesta-Frau,et al.  Characterization of Artifact Influence on the Classification of Glucose Time Series Using Sample Entropy Statistics , 2018, Entropy.

[27]  Hamed Azami,et al.  Amplitude-aware permutation entropy: Illustration in spike detection and signal segmentation , 2016, Comput. Methods Programs Biomed..

[28]  S. Swiryn,et al.  Predicting the onset of paroxysmal atrial fibrillation: the Computers in Cardiology Challenge 2001 , 2001, Computers in Cardiology 2001. Vol.28 (Cat. No.01CH37287).

[29]  Dingchang Zheng,et al.  Analysis of heart rate variability using fuzzy measure entropy , 2013, Comput. Biol. Medicine.

[30]  Claudia Carricarte Naranjo,et al.  Permutation entropy analysis of heart rate variability for the assessment of cardiovascular autonomic neuropathy in type 1 diabetes mellitus , 2017, Comput. Biol. Medicine.

[31]  David Cuesta-Frau,et al.  Patterns with Equal Values in Permutation Entropy: Do They Really Matter for Biosignal Classification? , 2018, Complex..

[32]  David Cuesta-Frau,et al.  Comparative study of approximate entropy and sample entropy robustness to spikes , 2011, Artif. Intell. Medicine.

[33]  Miguel A. F. Sanjuán,et al.  Combinatorial detection of determinism in noisy time series , 2008 .

[34]  Elif Derya Übeyli,et al.  Adaptive neuro-fuzzy inference system for classification of EEG signals using wavelet coefficients , 2005, Journal of Neuroscience Methods.

[35]  A. Vannucci,et al.  BICS Bath Institute for Complex Systems A note on time-dependent DiPerna-Majda measures , 2008 .

[36]  S M Pincus,et al.  Approximate entropy as a measure of system complexity. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Weiting Chen,et al.  Measuring complexity using FuzzyEn, ApEn, and SampEn. , 2009, Medical engineering & physics.

[38]  Massimiliano Zanin,et al.  Permutation Entropy and Its Main Biomedical and Econophysics Applications: A Review , 2012, Entropy.

[39]  David Cuesta-Frau,et al.  Classification of glucose records from patients at diabetes risk using a combined permutation entropy algorithm , 2018, Comput. Methods Programs Biomed..

[40]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[41]  Daryl R. DeFord,et al.  Random Walk Null Models for Time Series Data , 2017, Entropy.

[42]  Miguel A. F. Sanjuán,et al.  True and false forbidden patterns in deterministic and random dynamics , 2007 .

[43]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[44]  D. Bajic,et al.  The flip-flop effect in entropy estimation , 2011, 2011 IEEE 9th International Symposium on Intelligent Systems and Informatics.

[45]  Niels Wessel,et al.  Practical considerations of permutation entropy , 2013, The European Physical Journal Special Topics.

[46]  Leonardo Duque-Muñoz,et al.  Neuromuscular disease detection by neural networks and fuzzy entropy on time‐frequency analysis of electromyography signals , 2018, Expert Syst. J. Knowl. Eng..

[47]  Badong Chen,et al.  Weighted-permutation entropy: a complexity measure for time series incorporating amplitude information. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[48]  B. Pompe,et al.  Permutation entropy: a natural complexity measure for time series. , 2002, Physical review letters.

[49]  M. P. Griffin,et al.  Sample entropy analysis of neonatal heart rate variability. , 2002, American journal of physiology. Regulatory, integrative and comparative physiology.

[50]  K. Konstantinou,et al.  Reduction of randomness in seismic noise as a short-term precursor to a volcanic eruption , 2016, Scientific Reports.

[51]  Madalena Costa,et al.  Multiscale entropy analysis of biological signals. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  Ki H. Chon,et al.  Automatic Selection of the Threshold Value $r$ for Approximate Entropy , 2008, IEEE Transactions on Biomedical Engineering.

[53]  Yudong Zhang,et al.  Entropy Analysis of Short-Term Heartbeat Interval Time Series during Regular Walking , 2017, Entropy.

[54]  David Cuesta-Frau,et al.  Model Selection for Body Temperature Signal Classification Using Both Amplitude and Ordinality-Based Entropy Measures , 2018, Entropy.

[55]  Kemal Polat,et al.  Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform , 2007, Appl. Math. Comput..

[56]  Zhenhu Liang,et al.  Parameter selection in permutation entropy for an electroencephalographic measure of isoflurane anesthetic drug effect , 2013, Journal of Clinical Monitoring and Computing.

[57]  J. R. Moorman,et al.  Accurate estimation of entropy in very short physiological time series: the problem of atrial fibrillation detection in implanted ventricular devices. , 2011, American journal of physiology. Heart and circulatory physiology.