Stochastic encoding of graphs in deep learning allows for complex analysis of gender classification in resting-state and task functional brain networks from the UK Biobank

Classification of whole-brain functional connectivity MRI data with convolutional neural networks (CNNs) has shown promise, but the complexity of these models impedes understanding of which aspects of brain activity contribute to classification. While visualization techniques have been developed to interpret CNNs, bias inherent in the method of encoding abstract input data, as well as the natural variance of deep learning models, detract from the accuracy of these techniques. We introduce a stochastic encoding method in an ensemble of CNNs to classify functional connectomes by gender. We applied our method to resting-state and task data from the UK BioBank, using two visualization techniques to measure the salience of three brain networks involved in task- and resting-states, and their interaction. To regress confounding factors such as head motion, age, and intracranial volume, we introduced a multivariate balancing algorithm to ensure equal distributions of such covariates between classes in our data. We achieved a final AUROC of 0.8459. We found that resting-state data classifies more accurately than task data, with the inner salience network playing the most important role of the three networks overall in classification of resting-state data and connections to the central executive network in task data.

[1]  Jorge Sepulcre,et al.  Evidence from intrinsic activity that asymmetry of the human brain is controlled by multiple factors , 2009, Proceedings of the National Academy of Sciences.

[2]  R. Cabeza,et al.  Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies , 2000, Journal of Cognitive Neuroscience.

[3]  Nils M. Kriege,et al.  A survey on graph kernels , 2019, Applied Network Science.

[4]  A. Heinz,et al.  The Influence of Gender and Emotional Valence of Visual Cues on fMRI Activation in Humans , 2003, Pharmacopsychiatry.

[5]  Jennifer H. Pfeifer,et al.  Adolescent Gender Differences in Cognitive Control Performance and Functional Connectivity Between Default Mode and Fronto-Parietal Networks Within a Self-Referential Context , 2018, Front. Behav. Neurosci..

[6]  N. Makris,et al.  Sex Differences in Stress Response Circuitry Activation Dependent on Female Hormonal Cycle , 2010, The Journal of Neuroscience.

[7]  Markus Heinrichs,et al.  The neural correlates of sex differences in emotional reactivity and emotion regulation , 2009, Human brain mapping.

[8]  Joseph M. Andreano,et al.  Sex differences in the persistence of the amygdala response to negative material. , 2014, Social cognitive and affective neuroscience.

[9]  M. Mennes,et al.  Revisiting subcortical brain volume correlates of autism in the ABIDE dataset: effects of age and sex , 2017, Psychological Medicine.

[10]  G. Shulman,et al.  Medial prefrontal cortex and self-referential mental activity: Relation to a default mode of brain function , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[11]  Ruben C. Gur,et al.  Sex differences in brain and behavior in adolescence: Findings from the Philadelphia Neurodevelopmental Cohort , 2016, Neuroscience & Biobehavioral Reviews.

[12]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[13]  S. Baron-Cohen,et al.  Neuroscience and Biobehavioral Reviews a Meta-analysis of Sex Differences in Human Brain Structure , 2022 .

[14]  Yoshiro Okubo,et al.  Men and women show distinct brain activations during imagery of sexual and emotional infidelity , 2006, NeuroImage.

[15]  B. Mazoyer,et al.  Cortical networks for working memory and executive functions sustain the conscious resting state in man , 2001, Brain Research Bulletin.

[16]  P. McGuire,et al.  Functional atlas of emotional faces processing: a voxel-based meta-analysis of 105 functional magnetic resonance imaging studies. , 2009, Journal of psychiatry & neuroscience : JPN.

[17]  Edward T. Bullmore,et al.  Full Length Articles , 2022 .

[18]  Karine Sergerie,et al.  The role of the amygdala in emotional processing: A quantitative meta-analysis of functional neuroimaging studies , 2008, Neuroscience & Biobehavioral Reviews.

[19]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[20]  G. Glover,et al.  Dissociable Intrinsic Connectivity Networks for Salience Processing and Executive Control , 2007, The Journal of Neuroscience.

[21]  Mert R. Sabuncu,et al.  3D Convolutional Neural Networks for Classification of Functional Connectomes , 2018, DLMIA/ML-CDS@MICCAI.

[22]  Jure Leskovec,et al.  Representation Learning on Graphs: Methods and Applications , 2017, IEEE Data Eng. Bull..

[23]  Axel Schäfer,et al.  Gender differences in the processing of disgust- and fear-inducing pictures: an fMRI study , 2005, Neuroreport.

[24]  Ghassan Hamarneh,et al.  BrainNetCNN: Convolutional neural networks for brain networks; towards predicting neurodevelopment , 2017, NeuroImage.

[25]  C. Beckmann,et al.  Resting-state functional connectivity in major depressive disorder: A review , 2015, Neuroscience & Biobehavioral Reviews.

[26]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[27]  Efstathios D. Gennatas,et al.  Linked Sex Differences in Cognition and Functional Connectivity in Youth. , 2015, Cerebral cortex.

[28]  R Casanova,et al.  Combining Graph and Machine Learning Methods to Analyze Differences in Functional Connectivity Across Sex , 2012, The open neuroimaging journal.

[29]  Pascal Vincent,et al.  Visualizing Higher-Layer Features of a Deep Network , 2009 .

[30]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[31]  Yotam Hechtlinger,et al.  A Generalization of Convolutional Neural Networks to Graph-Structured Data , 2017, ArXiv.

[32]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[33]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  Stephan Felber,et al.  Gender differences in regional cerebral activity during the perception of emotion: A functional MRI study , 2006, NeuroImage.

[35]  Jared A. Nielsen,et al.  An Evaluation of the Left-Brain vs. Right-Brain Hypothesis with Resting State Functional Connectivity Magnetic Resonance Imaging , 2013, PloS one.

[36]  Dustin Scheinost,et al.  Task-induced brain state manipulation improves prediction of individual traits , 2018, Nature Communications.

[37]  Mark E Bastin,et al.  Sex Differences in the Adult Human Brain: Evidence from 5216 UK Biobank Participants , 2017, bioRxiv.

[38]  Issidoros C. Sarinopoulos,et al.  The effect of anticipation and the specificity of sex differences for amygdala and hippocampus function in emotional memory , 2006, Proceedings of the National Academy of Sciences.

[39]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[40]  H. Flor,et al.  Gender differences in the processing of standardized emotional visual stimuli in humans: a functional magnetic resonance imaging study , 2003, Neuroscience Letters.

[41]  Wei Gao,et al.  Task‐related modulation of functional connectivity variability and its behavioral correlations , 2015, Human brain mapping.

[42]  C. I. Wright,et al.  Sex-differential brain activation during exposure to female and male faces , 2004, Neuroreport.

[43]  M. Raichle,et al.  Emotion-induced changes in human medial prefrontal cortex: I. During cognitive task performance. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[44]  Yaoxue Zhang,et al.  Brain Differences Between Men and Women: Evidence From Deep Learning , 2019, Front. Neurosci..

[45]  Stephen M Smith,et al.  Correspondence of the brain's functional architecture during activation and rest , 2009, Proceedings of the National Academy of Sciences.

[46]  Ghassan Hamarneh,et al.  Connectome priors in deep neural networks to predict autism , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[47]  Michalis Vazirgiannis,et al.  Kernel Graph Convolutional Neural Networks , 2017, ICANN.

[48]  John Suckling,et al.  A wavelet method for modeling and despiking motion artifacts from resting-state fMRI time series , 2014, NeuroImage.

[49]  Stephen D. Mayhew,et al.  Gender Specific Re-organization of Resting-State Networks in Older Age , 2016, Front. Aging Neurosci..

[50]  S. Hamann,et al.  Men and women differ in amygdala response to visual sexual stimuli , 2004, Nature Neuroscience.

[51]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[52]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[53]  Vince D. Calhoun,et al.  Lateralization of resting state networks and relationship to age and gender , 2015, NeuroImage.

[54]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[55]  Natalia S. Lawrence,et al.  Sex Differences in Neural Responses to Disgusting Visual Stimuli: Implications for Disgust-Related Psychiatric Disorders , 2007, Biological Psychiatry.

[56]  A. Aleman,et al.  Sex Differences in Neural Activation to Facial Expressions Denoting Contempt and Disgust , 2008, PloS one.

[57]  Francesco Fera,et al.  The Amygdala Response to Emotional Stimuli: A Comparison of Faces and Scenes , 2002, NeuroImage.

[58]  Daoqiang Zhang,et al.  Topological graph kernel on multiple thresholded functional connectivity networks for mild cognitive impairment classification , 2014, Human brain mapping.

[59]  R. Bluhm,et al.  Default mode network connectivity: effects of age, sex, and analytic approach , 2008, Neuroreport.

[60]  M. Corbetta,et al.  Common Blood Flow Changes across Visual Tasks: II. Decreases in Cerebral Cortex , 1997, Journal of Cognitive Neuroscience.

[61]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[62]  Michalis Vazirgiannis,et al.  Classifying Graphs as Images with Convolutional Neural Networks , 2017, ArXiv.

[63]  John Suckling,et al.  Ensemble Deep Learning on Large, Mixed-Site fMRI Datasets in Autism and Other Tasks , 2020, Int. J. Neural Syst..

[64]  Justin L. Vincent,et al.  Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[65]  G. Fink,et al.  Dorsal and Ventral Attention Systems : Distinct Neural Circuits but Collaborative Roles , 2013 .

[66]  J. Binder,et al.  A Parametric Manipulation of Factors Affecting Task-induced Deactivation in Functional Neuroimaging , 2003, Journal of Cognitive Neuroscience.

[67]  Abraham Z. Snyder,et al.  Function in the human connectome: Task-fMRI and individual differences in behavior , 2013, NeuroImage.

[68]  Maurizio Corbetta,et al.  The human brain is intrinsically organized into dynamic, anticorrelated functional networks. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[69]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[70]  K. Davis,et al.  Cognitive and default‐mode resting state networks: Do male and female brains “rest” differently? , 2010, Human brain mapping.

[71]  Fenna M. Krienen,et al.  Opportunities and limitations of intrinsic functional connectivity MRI , 2013, Nature Neuroscience.

[72]  V. Menon,et al.  A critical role for the right fronto-insular cortex in switching between central-executive and default-mode networks , 2008, Proceedings of the National Academy of Sciences.

[73]  V. Calhoun,et al.  Machine learning of brain gray matter differentiates sex in a large forensic sample , 2018, Human brain mapping.

[74]  Monique Ernst,et al.  A developmental examination of gender differences in brain engagement during evaluation of threat , 2004, Biological Psychiatry.

[75]  Sebastian Scherer,et al.  VoxNet: A 3D Convolutional Neural Network for real-time object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[76]  N. Volkow,et al.  Association between functional connectivity hubs and brain networks. , 2011, Cerebral cortex.

[77]  Jeffrey S. Anderson,et al.  Local brain connectivity and associations with gender and age , 2011, Developmental Cognitive Neuroscience.

[78]  Jonathan P. McNulty,et al.  The salience network is responsible for switching between the default mode network and the central executive network: Replication from DCM , 2014, NeuroImage.

[79]  B. Turetsky,et al.  An fMRI Study of Sex Differences in Regional Activation to a Verbal and a Spatial Task , 2000, Brain and Language.

[80]  Fei-Fei Li,et al.  Deep visual-semantic alignments for generating image descriptions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[81]  K. Hugdahl,et al.  Sex differences in visuo-spatial processing: An fMRI study of mental rotation , 2006, Neuropsychologia.