Spatio-Temporal Graph Convolution for Resting-State fMRI Analysis

The Blood-Oxygen-Level-Dependent (BOLD) signal of resting-state fMRI (rs-fMRI) records the temporal dynamics of intrinsic functional networks in the brain. However, existing deep learning methods applied to rs-fMRI either neglect the functional dependency between different brain regions in a network or discard the information in the temporal dynamics of brain activity. To overcome those shortcomings, we propose to formulate functional connectivity networks within the context of spatio-temporal graphs. We train a spatio-temporal graph convolutional network (ST-GCN) on short sub-sequences of the BOLD time series to model the non-stationary nature of functional connectivity. Simultaneously, the model learns the importance of graph edges within ST-GCN to gain insight into the functional connectivities contributing to the prediction. In analyzing the rs-fMRI of the Human Connectome Project (HCP, N = 1,091) and the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA, N = 773), ST-GCN is significantly more accurate than common approaches in predicting gender and age based on BOLD signals. Furthermore, the brain regions and functional connections significantly contributing to the predictions of our model are important markers according to the neuroscience literature.

[1]  Ben Glocker,et al.  Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks , 2017, MICCAI.

[2]  Jesper Andersson,et al.  A multi-modal parcellation of human cerebral cortex , 2016, Nature.

[3]  Eswar Damaraju,et al.  Tracking whole-brain connectivity dynamics in the resting state. , 2014, Cerebral cortex.

[4]  Yong Fan,et al.  Brain age prediction based on resting-state functional connectivity patterns using convolutional neural networks , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[5]  A. Dale,et al.  Life-span changes of the human brain white matter: diffusion tensor imaging (DTI) and volumetry. , 2010, Cerebral cortex.

[6]  Simon B Eickhoff,et al.  Sex Classification by Resting State Brain Connectivity , 2019, bioRxiv.

[7]  Lei Guo,et al.  Identifying Brain Networks of Multiple Time Scales via Deep Recurrent Neural Network , 2018, MICCAI.

[8]  Torsten Rohlfing,et al.  Influences of Age, Sex, and Moderate Alcohol Drinking on the Intrinsic Functional Architecture of Adolescent Brains , 2018, Cerebral cortex.

[9]  Arno Klein,et al.  101 Labeled Brain Images and a Consistent Human Cortical Labeling Protocol , 2012, Front. Neurosci..

[10]  Qingyu Zhao,et al.  Longitudinally consistent estimates of intrinsic functional networks , 2019, Human brain mapping.

[11]  Thomas E. Nichols,et al.  Functional connectomics from resting-state fMRI , 2013, Trends in Cognitive Sciences.

[12]  Yong Fan,et al.  Brain Decoding from Functional MRI Using Long Short-Term Memory Recurrent Neural Networks , 2018, MICCAI.

[13]  Zhanxing Zhu,et al.  Spatio-temporal Graph Convolutional Neural Network: A Deep Learning Framework for Traffic Forecasting , 2017, IJCAI.

[14]  Yuan Zhou,et al.  Graph Neural Network for Interpreting Task-fMRI Biomarkers , 2019, MICCAI.

[15]  Kilian M. Pohl,et al.  Variational Autoencoder with Truncated Mixture of Gaussians for Functional Connectivity Analysis , 2019, IPMI.

[16]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[17]  Fenna M. Krienen,et al.  Opportunities and limitations of intrinsic functional connectivity MRI , 2013, Nature Neuroscience.

[18]  Pierre Bellec,et al.  Functional Annotation of Human Cognitive States using Graph Convolution Networks , 2019 .

[19]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[20]  Alex D. Leow,et al.  From Default Mode Network to the Basal Configuration: Sex Differences in the Resting-State Brain Connectivity as a Function of Age and Their Clinical Correlates , 2018, Front. Psychiatry.

[21]  Torsten Rohlfing,et al.  The National Consortium on Alcohol and NeuroDevelopment in Adolescence (NCANDA): A Multisite Study of Adolescent Development and Substance Use. , 2015, Journal of studies on alcohol and drugs.

[22]  Jalil Taghia,et al.  Bayesian switching factor analysis for estimating time-varying functional connectivity in fMRI , 2017, NeuroImage.

[23]  Xiang Li,et al.  Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN) , 2018, MICCAI.

[24]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[25]  Bertrand Thirion,et al.  Functional annotation of human cognitive states using deep graph convolution , 2020, NeuroImage.

[26]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[27]  Kilian M. Pohl,et al.  Confounder-Aware Visualization of ConvNets , 2019, MLMI@MICCAI.

[28]  Nicha C. Dvornek,et al.  Identifying Autism from Resting-State fMRI Using Long Short-Term Memory Networks , 2017, MLMI@MICCAI.

[29]  Anthony A Grace,et al.  Inhibitory Modulation of Orbitofrontal Cortex on Medial Prefrontal Cortex–Amygdala Information Flow , 2018, Cerebral cortex.

[30]  Habib Benali,et al.  Partial correlation for functional brain interactivity investigation in functional MRI , 2006, NeuroImage.

[31]  Mark Jenkinson,et al.  The minimal preprocessing pipelines for the Human Connectome Project , 2013, NeuroImage.

[32]  Jiening Zhan,et al.  Temporal Graph Convolutional Networks for Automatic Seizure Detection , 2019, MLHC.