Modeling Multivariate Biosignals With Graph Neural Networks and Structured State Space Models

Multivariate biosignals are prevalent in many medical domains, such as electroencephalography, polysomnography, and electrocardiography. Modeling spatiotemporal dependencies in multivariate biosignals is challenging due to (1) long-range temporal dependencies and (2) complex spatial correlations between the electrodes. To address these challenges, we propose representing multivariate biosignals as time-dependent graphs and introduce GraphS4mer, a general graph neural network (GNN) architecture that improves performance on biosignal classification tasks by modeling spatiotemporal dependencies in biosignals. Specifically, (1) we leverage the Structured State Space architecture, a state-of-the-art deep sequence model, to capture long-range temporal dependencies in biosignals and (2) we propose a graph structure learning layer in GraphS4mer to learn dynamically evolving graph structures in the data. We evaluate our proposed model on three distinct biosignal classification tasks and show that GraphS4mer consistently improves over existing models, including (1) seizure detection from electroencephalographic signals, outperforming a previous GNN with self-supervised pre-training by 3.1 points in AUROC; (2) sleep staging from polysomnographic signals, a 4.1 points improvement in macro-F1 score compared to existing sleep staging models; and (3) 12-lead electrocardiogram classification, outperforming previous state-of-the-art models by 2.7 points in macro-F1 score.

[1]  G. Wingen,et al.  Improving the Diagnosis of Psychiatric Disorders with Self-Supervised Graph State Space Models , 2022, ArXiv.

[2]  Daniel Y. Fu,et al.  FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness , 2022, NeurIPS.

[3]  Shikun Liu,et al.  Spatio-Temporal Latent Graph Structure Learning for Traffic Forecasting , 2022, 2022 International Joint Conference on Neural Networks (IJCNN).

[4]  Albert Gu,et al.  It's Raw! Audio Generation with State-Space Models , 2022, ICML.

[5]  Kaare B. Mikkelsen,et al.  Automatic sleep staging of EEG signals: recent development, challenges, and future directions , 2021, Physiological measurement.

[6]  Albert Gu,et al.  Efficiently Modeling Long Sequences with Structured State Spaces , 2021, ICLR.

[7]  Atri Rudra,et al.  Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers , 2021, NeurIPS.

[8]  M. Zitnik,et al.  Graph-Guided Network for Irregularly Sampled Multivariate Time Series , 2021, ICLR.

[9]  Guido A. van Wingen,et al.  Dynamic Adaptive Spatio-Temporal Graph Convolution for fMRI Modelling , 2021, MLCN@MICCAI.

[10]  Jared A. Dunnmon,et al.  Self-Supervised Graph Neural Networks for Improved Electroencephalographic Seizure Analysis , 2021, ICLR.

[11]  Wai Kin Victor Chan,et al.  Spatial‐temporal attention wavenet: A deep learning framework for traffic prediction considering spatial‐temporal dependencies , 2021, IET Intelligent Transport Systems.

[12]  Carl Yang,et al.  A Survey on Graph Structure Learning: Progress and Opportunities , 2021, 2103.03036.

[13]  Weiwei Jiang,et al.  Graph Neural Network for Traffic Forecasting: A Survey , 2021, Expert Syst. Appl..

[14]  J. Bi,et al.  Discrete Graph Structure Learning for Forecasting Multiple Time Series , 2021, ICLR.

[15]  V. Thorey,et al.  RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale , 2021, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Bo Zong,et al.  Learning to Drop: Robust Graph Neural Network via Topological Denoising , 2020, WSDM.

[17]  Liu Yang,et al.  Long Range Arena: A Benchmark for Efficient Transformers , 2020, ICLR.

[18]  Lucy J. Colwell,et al.  Rethinking Attention with Performers , 2020, ICLR.

[19]  Michalis Vazirgiannis,et al.  Transfer Graph Neural Networks for Pandemic Forecasting , 2020, AAAI.

[20]  C. Ré,et al.  HiPPO: Recurrent Memory with Optimal Polynomial Projections , 2020, NeurIPS.

[21]  Bo Zong,et al.  Robust Graph Representation Learning via Neural Sparsification , 2020, ICML.

[22]  Luyang Liu,et al.  Examining COVID-19 Forecasting using Spatio-Temporal Graph Neural Networks , 2020, ArXiv.

[23]  Lina Yao,et al.  Adaptive Graph Convolutional Recurrent Network for Traffic Forecasting , 2020, NeurIPS.

[24]  Amir F. Atiya,et al.  Epileptic Seizures Detection Using Deep Learning Techniques: A Review , 2020, International journal of environmental research and public health.

[25]  Jing Wang,et al.  GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification , 2020, IJCAI.

[26]  Nikolaos Pappas,et al.  Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention , 2020, ICML.

[27]  Chen Cai,et al.  A Note on Over-Smoothing for Graph Neural Networks , 2020, ArXiv.

[28]  Omri Azencot,et al.  Lipschitz Recurrent Neural Networks , 2020, ICLR.

[29]  Davide Eynard,et al.  Temporal Graph Networks for Deep Learning on Dynamic Graphs , 2020, ArXiv.

[30]  Xiang Zhang,et al.  GNNGuard: Defending Graph Neural Networks against Adversarial Attacks , 2020, NeurIPS.

[31]  Mohammed J. Zaki,et al.  Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings , 2020, NeurIPS.

[32]  Suhang Wang,et al.  Graph Structure Learning for Robust Graph Neural Networks , 2020, KDD.

[33]  Christopher Ré,et al.  Machine Learning on Graphs: A Model and Comprehensive Taxonomy , 2020, J. Mach. Learn. Res..

[34]  Wojciech Samek,et al.  Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL , 2020, IEEE Journal of Biomedical and Health Informatics.

[35]  Jared A. Dunnmon,et al.  Weak supervision as an efficient approach for automated seizure detection in electroencephalography , 2020, npj Digital Medicine.

[36]  Gaofeng Meng,et al.  Spatio-Temporal Graph Structure Learning for Traffic Forecasting , 2020, AAAI.

[37]  Zhang Ming,et al.  Graph neural network , 2020 .

[38]  Da Xu,et al.  Inductive Representation Learning on Temporal Graphs , 2020, ICLR.

[39]  Lukasz Kaiser,et al.  Reformer: The Efficient Transformer , 2020, ICLR.

[40]  Lars Petersson,et al.  Neural Memory Networks for Robust Classification of Seizure Type , 2019, ArXiv.

[41]  Cheng Wang,et al.  GMAN: A Graph Multi-Attention Network for Traffic Prediction , 2019, AAAI.

[42]  V. Thorey,et al.  Dreem Open Datasets: Multi-Scored Sleep Datasets to Compare Human and Automated Sleep Staging , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[43]  Geoffrey I. Webb,et al.  InceptionTime: Finding AlexNet for time series classification , 2019, Data Mining and Knowledge Discovery.

[44]  Jing Jiang,et al.  Graph WaveNet for Deep Spatial-Temporal Graph Modeling , 2019, IJCAI.

[45]  J. Leskovec,et al.  Strategies for Pre-training Graph Neural Networks , 2019, ICLR.

[46]  Hong Cheng,et al.  Predicting Path Failure In Time-Evolving Graphs , 2019, KDD.

[47]  Jiening Zhan,et al.  Temporal Graph Convolutional Networks for Automatic Seizure Detection , 2019, MLHC.

[48]  Wei Hu,et al.  Exploring Structure-Adaptive Graph Learning for Robust Semi-Supervised Classification , 2019, 2020 IEEE International Conference on Multimedia and Expo (ICME).

[49]  Ilya Sutskever,et al.  Generating Long Sequences with Sparse Transformers , 2019, ArXiv.

[50]  Jianbin Tang,et al.  SeizureNet: Multi-Spectral Deep Feature Learning for Seizure Type Classification , 2019, MLCN/RNO-AI@MICCAI.

[51]  Charles E. Leisersen,et al.  EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs , 2019, AAAI.

[52]  Jennifer L. Martin,et al.  Clinical Use of a Home Sleep Apnea Test: An Updated American Academy of Sleep Medicine Position Statement. , 2018, Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine.

[53]  Zhi Zhang,et al.  Bag of Tricks for Image Classification with Convolutional Neural Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[54]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[55]  Shoushui Wei,et al.  An Open Access Database for Evaluating the Algorithms of Electrocardiogram Rhythm and Morphology Abnormality Detection , 2018, Journal of Medical Imaging and Health Informatics.

[56]  Ruoyu Li,et al.  Adaptive Graph Convolutional Neural Networks , 2018, AAAI.

[57]  Joseph Picone,et al.  The Temple University Hospital Seizure Detection Corpus , 2018, Front. Neuroinform..

[58]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[59]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.

[60]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[61]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[62]  Chao Wu,et al.  DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-Channel EEG , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[63]  Xavier Bresson,et al.  Structured Sequence Modeling with Graph Convolutional Recurrent Networks , 2016, ICONIP.

[64]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[65]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[66]  Frank Hutter,et al.  SGDR: Stochastic Gradient Descent with Warm Restarts , 2016, ICLR.

[67]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[68]  Vassilis Kalofolias,et al.  How to Learn a Graph from Smooth Signals , 2016, AISTATS.

[69]  Yoshua Bengio,et al.  Unitary Evolution Recurrent Neural Networks , 2015, ICML.

[70]  A. Garfinkel,et al.  Nonlinear and Stochastic Dynamics in the Heart. , 2014, Physics reports.

[71]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[72]  Tomas B. Garcia,et al.  12-Lead ECG: The Art of Interpretation , 2013 .

[73]  Christoph M. Michel,et al.  Towards the utilization of EEG as a brain imaging tool , 2012, NeuroImage.

[74]  Luc Berthouze,et al.  Human EEG shows long-range temporal correlations of oscillation amplitude in Theta, Alpha and Beta bands across a wide age range , 2010, Clinical Neurophysiology.

[75]  Frédérique Amor,et al.  Cortical local and long-range synchronization interplay in human absence seizure initiation , 2009, NeuroImage.

[76]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[77]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[78]  Jiayuan He,et al.  Hand Gesture Recognition and Biometric Authentication Using a Multi-day Dataset , 2022, ICIRA.

[79]  Jianfei Gao,et al.  On the Equivalence Between Temporal and Static Equivariant Graph Representations , 2022, ICML.

[80]  Mona Sazgar,et al.  Overview of EEG, Electrode Placement, and Montages , 2019, Absolute Epilepsy and EEG Rotation Review.

[81]  A. Chesson,et al.  The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology, and Techinical Specifications , 2007 .

[82]  Jeffrey M. Hausdorff,et al.  Fractal mechanisms and heart rate dynamics. Long-range correlations and their breakdown with disease. , 1995, Journal of electrocardiology.

[83]  George K. Kostopoulos,et al.  Thalamocortical Relationships in Generalized Epilepsy with Bilaterally Synchronous Spike-and-Wave Discharge , 1990 .