Differentiable neural architecture search for optimal spatial/temporal brain function network decomposition

It has been a key topic to decompose the brain's spatial/temporal function networks from 4D functional magnetic resonance imaging (fMRI) data. With the advantages of robust and meaningful brain pattern extraction, deep neural networks have been shown to be more powerful and flexible in fMRI data modeling than other traditional methods. However, the challenge of designing neural network architecture for high-dimensional and complex fMRI data has also been realized recently. In this paper, we propose a new spatial/temporal differentiable neural architecture search algorithm (ST-DARTS) for optimal brain network decomposition. The core idea of ST-DARTS is to optimize the inner cell structure of the vanilla recurrent neural network (RNN) in order to effectively decompose spatial/temporal brain function networks from fMRI data. Based on the evaluations on all seven fMRI tasks in human connectome project (HCP) dataset, the ST-DARTS model is shown to perform promisingly, both spatially (i.e., it can recognize the most stimuli-correlated spatial brain network activation that is very similar to the benchmark) and temporally (i.e., its temporal activity is highly positively correlated with the task-design). To further improve the efficiency of ST-DARTS model, we introduce a flexible early-stopping mechanism, named as ST-DARTS+, which further improves experimental results significantly. To our best knowledge, the proposed ST-DARTS and ST-DARTS+ models are among the early efforts in optimally decomposing spatial/temporal brain function networks from fMRI data with neural architecture search strategy and they demonstrate great promise.

[1]  Ramesh Raskar,et al.  Designing Neural Network Architectures using Reinforcement Learning , 2016, ICLR.

[2]  P. Downing,et al.  Within‐subject reproducibility of category‐specific visual activation with functional MRI , 2005, Human brain mapping.

[3]  Lorenzo Torresani,et al.  Connectivity Learning in Multi-Branch Networks , 2017, ArXiv.

[4]  Frank Hutter,et al.  Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution , 2018, ICLR.

[5]  Vince D. Calhoun,et al.  Restricted Boltzmann machines for neuroimaging: An application in identifying intrinsic networks , 2014, NeuroImage.

[6]  J. Ford,et al.  Default mode network activity and connectivity in psychopathology. , 2012, Annual review of clinical psychology.

[7]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[8]  Mariko Osaka,et al.  Coactivation of the Default Mode Network regions and Working Memory Network regions during task preparation , 2014, Scientific Reports.

[9]  Wei Zhang,et al.  Connectome-scale functional intrinsic connectivity networks in macaques , 2017, Neuroscience.

[10]  Tianming Liu,et al.  Modeling task-based fMRI data via deep belief network with neural architecture search , 2020, Comput. Medical Imaging Graph..

[11]  Jinglei Lv,et al.  Experimental Comparisons of Sparse Dictionary Learning and Independent Component Analysis for Brain Network Inference From fMRI Data , 2019, IEEE Transactions on Biomedical Engineering.

[12]  Alok Aggarwal,et al.  Regularized Evolution for Image Classifier Architecture Search , 2018, AAAI.

[13]  Walter J. Scheirer,et al.  A Neurobiological Evaluation Metric for Neural Network Model Search , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Elliot Meyerson,et al.  Evolving Deep Neural Networks , 2017, Artificial Intelligence in the Age of Neural Networks and Brain Computing.

[15]  Quoc V. Le,et al.  Neural Architecture Search with Reinforcement Learning , 2016, ICLR.

[16]  Vince D. Calhoun,et al.  A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data , 2009, NeuroImage.

[17]  Li Yao,et al.  Multiple neural networks supporting a semantic task: An fMRI study using independent component analysis , 2009, NeuroImage.

[18]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[19]  Qing Li,et al.  Optimize CNN Model for FMRI Signal Classification Via Adanet-Based Neural Architecture Search , 2020, 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI).

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Teresa Paiva,et al.  Task positive and default mode networks during a parametric working memory task in obstructive sleep apnea patients and healthy controls. , 2011, Sleep.

[22]  Yiming Yang,et al.  DARTS: Differentiable Architecture Search , 2018, ICLR.

[23]  Mark W. Woolrich,et al.  Fully Bayesian spatio-temporal modeling of FMRI data , 2004, IEEE Transactions on Medical Imaging.

[24]  Jeffrey D Schall,et al.  The role of working memory representations in the control of attention. , 2007, Cerebral cortex.

[25]  Quoc V. Le,et al.  Efficient Neural Architecture Search via Parameter Sharing , 2018, ICML.

[26]  Walter Schneider,et al.  A rapid fMRI task battery for mapping of visual, motor, cognitive, and emotional function , 2006, NeuroImage.

[27]  Wei Zhang,et al.  Automatic Recognition of fMRI-Derived Functional Networks Using 3-D Convolutional Neural Networks , 2018, IEEE Transactions on Biomedical Engineering.

[28]  Jinglei Lv,et al.  Modeling Hierarchical Brain Networks via Volumetric Sparse Deep Belief Network , 2020, IEEE Transactions on Biomedical Engineering.

[29]  J. Lv,et al.  Assessing effects of prenatal alcohol exposure using group-wise sparse representation of fMRI data , 2015, Psychiatry Research: Neuroimaging.

[30]  Qing Li,et al.  Identify Hierarchical Structures from Task-Based fMRI Data via Hybrid Spatiotemporal Neural Architecture Search Net , 2019, MICCAI.

[31]  Qi Tian,et al.  Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[32]  Xiang Li,et al.  Spatio-temporal modeling of connectome-scale brain network interactions via time-evolving graphs , 2017, NeuroImage.

[33]  Shawn C. Milleville,et al.  Understanding Animate Agents , 2007, Psychological science.

[34]  Quoc V. Le,et al.  Large-Scale Evolution of Image Classifiers , 2017, ICML.

[35]  Xiang Li,et al.  Modeling 4D fMRI Data via Spatio-Temporal Convolutional Neural Networks (ST-CNN) , 2018, MICCAI.

[36]  J. Pekar,et al.  A method for making group inferences from functional MRI data using independent component analysis , 2001, Human brain mapping.

[37]  Dawn Xiaodong Song,et al.  Differentiable Neural Network Architecture Search , 2018, ICLR.

[38]  Mary E. Meyerand,et al.  The effect of scan length on the reliability of resting-state fMRI connectivity estimates , 2013, NeuroImage.

[39]  Feng Liu,et al.  Deep Learning and Its Applications in Biomedicine , 2018, Genom. Proteom. Bioinform..

[40]  Stephen M. Smith,et al.  General multilevel linear modeling for group analysis in FMRI , 2003, NeuroImage.

[41]  Gaofeng Meng,et al.  RENAS: Reinforced Evolutionary Neural Architecture Search , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Maurizio Corbetta,et al.  The human brain is intrinsically organized into dynamic, anticorrelated functional networks. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Xuanjing Huang,et al.  Bridging LSTM Architecture and the Neural Dynamics during Reading , 2016, IJCAI.

[44]  Qing Li,et al.  Neural Architecture Search for Optimization of Spatial-Temporal Brain Network Decomposition , 2020, MICCAI.

[45]  V. Calhoun,et al.  Multisubject Independent Component Analysis of fMRI: A Decade of Intrinsic Networks, Default Mode, and Neurodiagnostic Discovery , 2012, IEEE Reviews in Biomedical Engineering.

[46]  Jieping Ye,et al.  Holistic Atlases of Functional Networks and Interactions Reveal Reciprocal Organizational Architecture of Cortical Function , 2015, IEEE Transactions on Biomedical Engineering.

[47]  Stephen D. Mayhew,et al.  Spatial–temporal modelling of fMRI data through spatially regularized mixture of hidden process models , 2014, NeuroImage.

[48]  Yu Zhao,et al.  Supervised Dictionary Learning for Inferring Concurrent Brain Networks , 2015, IEEE Transactions on Medical Imaging.

[49]  Li Yao,et al.  A Triple Network Connectivity Study of Large-Scale Brain Systems in Cognitively Normal APOE4 Carriers , 2016, Front. Aging Neurosci..

[50]  Thomas Brox,et al.  Understanding and Robustifying Differentiable Architecture Search , 2020, ICLR.

[51]  Alberto Signoroni,et al.  CEREBRUM: a fast and fully-volumetric Convolutional Encoder-decodeR for weakly-supervised sEgmentation of BRain strUctures from out-of-the-scanner MRI , 2019, Medical Image Anal..

[52]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[53]  Tomoyasu Horikawa,et al.  Generic decoding of seen and imagined objects using hierarchical visual features , 2015, Nature Communications.

[54]  Cameron S. Carter,et al.  Brain Is to Thought as Stomach Is to ??: Investigating the Role of Rostrolateral Prefrontal Cortex in Relational Reasoning , 2008, Journal of Cognitive Neuroscience.

[55]  Lei Guo,et al.  Correction to: Gyral-sulcal contrast in intrinsic functional brain networks across task performances , 2020, Brain Imaging and Behavior.

[56]  Yu Zhao,et al.  Modeling Task fMRI Data Via Deep Convolutional Autoencoder , 2018, IEEE Transactions on Medical Imaging.

[57]  J. Morris,et al.  Evaluation of different techniques for detection of virulence in Yersinia enterocolitica , 1990, Journal of Clinical Microbiology.

[58]  Gulshan Kumar,et al.  A Survey of Deep Learning and Its Applications: A New Paradigm to Machine Learning , 2019, Archives of Computational Methods in Engineering.

[59]  Wei Zhang,et al.  Task fMRI data analysis based on supervised stochastic coordinate coding , 2017, Medical Image Anal..

[60]  Ramesh Raskar,et al.  Accelerating Neural Architecture Search using Performance Prediction , 2017, ICLR.

[61]  C. Degueldre,et al.  Regional brain activity during working memory tasks. , 1996, Brain : a journal of neurology.

[62]  Heng Huang,et al.  Sparse representation of whole-brain fMRI signals for identification of functional networks , 2015, Medical Image Anal..

[63]  Song Han,et al.  Path-Level Network Transformation for Efficient Architecture Search , 2018, ICML.

[64]  Han Wang,et al.  Recognizing Brain States Using Deep Sparse Recurrent Neural Network , 2019, IEEE Transactions on Medical Imaging.

[65]  Abraham Z. Snyder,et al.  Function in the human connectome: Task-fMRI and individual differences in behavior , 2013, NeuroImage.

[66]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[67]  Yu Zhao,et al.  Simultaneous Spatial-Temporal Decomposition of Connectome-Scale Brain Networks by Deep Sparse Recurrent Auto-Encoders , 2019, IPMI.

[68]  Kurt E. Weaver,et al.  Mapping anterior temporal lobe language areas with fMRI: A multicenter normative study , 2011, NeuroImage.

[69]  A. Mechelli,et al.  Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications , 2017, Neuroscience & Biobehavioral Reviews.

[70]  Yong Yu,et al.  Efficient Architecture Search by Network Transformation , 2017, AAAI.

[71]  Nikolaus Kriegeskorte,et al.  Deep neural networks: a new framework for modelling biological vision and brain information processing , 2015, bioRxiv.

[72]  L. R. Dice.  Measures of the Amount of Ecologic Association Between Species , 1945 .

[73]  Bo Peng,et al.  Latent source mining in FMRI via restricted Boltzmann machine , 2018, Human brain mapping.