DeepCSR: A 3D Deep Learning Approach for Cortical Surface Reconstruction

The study of neurodegenerative diseases relies on the reconstruction and analysis of the brain cortex from magnetic resonance imaging (MRI). Traditional frameworks for this task like FreeSurfer demand lengthy runtimes, while its accelerated variant FastSurfer still relies on a voxel-wise segmentation which is limited by its resolution to capture narrow continuous objects as cortical surfaces. Having these limitations in mind, we propose DeepCSR, a 3D deep learning framework for cortical surface reconstruction from MRI. Towards this end, we train a neural network model with hypercolumn features to predict implicit surface representations for points in a brain template space. After training, the cortical surface at a desired level of detail is obtained by evaluating surface representations at specific coordinates, and subsequently applying a topology correction algorithm and an isosurface extraction method. Thanks to the continuous nature of this approach and the efficacy of its hypercolumn features scheme, DeepCSR efficiently reconstructs cortical surfaces at high resolution capturing fine details in the cortical folding. Moreover, DeepCSR is as accurate, more precise, and faster than the widely used FreeSurfer toolbox and its deep learning powered variant FastSurfer on reconstructing cortical surfaces from MRI which should facilitate large-scale medical studies and new healthcare applications.

[1]  Meritxell Bach Cuadra,et al.  A Surface-Based Approach to Quantify Local Cortical Gyrification , 2008, IEEE Transactions on Medical Imaging.

[2]  Leonidas J. Guibas,et al.  A metric for distributions with applications to image databases , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Colin Studholme,et al.  A framework for in vivo quantification of regional brain folding in premature neonates , 2008, NeuroImage.

[4]  J. Pariente,et al.  Early diagnosis of Alzheimer's disease using cortical thickness: impact of cognitive reserve , 2009, Brain : a journal of neurology.

[5]  Richard M. Leahy,et al.  BrainSuite: An Automated Cortical Surface Identification Tool , 2000, MICCAI.

[6]  Alan C. Evans,et al.  Automated 3-D extraction and evaluation of the inner and outer cortical surfaces using a Laplacian map and partial volume effect classification , 2005, NeuroImage.

[7]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[8]  Xiao Han,et al.  CRUISE: Cortical reconstruction using implicit surface evolution , 2004, NeuroImage.

[9]  Mathieu Aubry,et al.  AtlasNet: A Papier-M\^ach\'e Approach to Learning 3D Surface Generation , 2018, CVPR 2018.

[10]  Olivier Salvado,et al.  Going Deeper With Brain Morphometry Using Neural Networks , 2020, 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI).

[11]  Sebastian Nowozin,et al.  Occupancy Networks: Learning 3D Reconstruction in Function Space , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Arthur W. Toga,et al.  A Probabilistic Atlas of the Human Brain: Theory and Rationale for Its Development The International Consortium for Brain Mapping (ICBM) , 1995, NeuroImage.

[13]  Yinda Zhang,et al.  Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  John G. Csernansky,et al.  Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI Data in Young, Middle Aged, Nondemented, and Demented Older Adults , 2007, Journal of Cognitive Neuroscience.

[15]  M. Weissman,et al.  Test–retest reliability of freesurfer measurements within and between sites: Effects of visual approval process , 2015, Human brain mapping.

[16]  Rainer Goebel,et al.  An Efficient Algorithm for Topologically Correct Segmentation of the Cortical Sheet in Anatomical MR Volumes , 2001, NeuroImage.

[17]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[18]  Thomas Lewiner,et al.  Efficient Implementation of Marching Cubes' Cases with Topological Guarantees , 2003, J. Graphics, GPU, & Game Tools.

[19]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Jurgen Fripp,et al.  Cross-sectional and longitudinal analysis of the relationship between Aβ deposition, cortical thickness, and memory in cognitively unimpaired individuals and in Alzheimer disease. , 2013, JAMA neurology.

[21]  Aaron C. Courville,et al.  Learning Visual Reasoning Without Strong Priors , 2017, ICML 2017.

[22]  Chen Kong,et al.  Using Locally Corresponding CAD Models for Dense 3D Reconstructions from a Single Image , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jerry L. Prince,et al.  Digital Topology in Brain Imaging , 2010, IEEE Signal Processing Magazine.

[24]  Christian Gaser,et al.  Cortical thickness and central surface estimation , 2013, NeuroImage.

[25]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[26]  Daniel P. Huttenlocher,et al.  Comparing Images Using the Hausdorff Distance , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Peter V. Gehler,et al.  Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation , 2018, 2018 International Conference on 3D Vision (3DV).

[28]  Alan C. Evans,et al.  Automated 3-D Extraction of Inner and Outer Surfaces of Cerebral Cortex from MRI , 2000, NeuroImage.

[29]  Andrew Zisserman,et al.  Estimation of the partial volume effect in MRI , 2002, Medical Image Anal..

[30]  Nick C Fox,et al.  The Alzheimer's disease neuroimaging initiative (ADNI): MRI methods , 2008, Journal of magnetic resonance imaging : JMRI.

[31]  Wei Liu,et al.  Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images , 2018, ECCV.

[32]  B. Fischl,et al.  FastSurfer - A fast and accurate deep learning based neuroimaging pipeline , 2019, NeuroImage.

[33]  Allan Hanbury,et al.  Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool , 2015, BMC Medical Imaging.

[34]  Mauricio Reyes,et al.  Brain Morphometry Estimation: From Hours to Seconds Using Deep Learning , 2020, Frontiers in Neurology.

[35]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[36]  Jerry L. Prince,et al.  Cortical reconstruction using implicit surface evolution: Accuracy and precision analysis , 2006, NeuroImage.

[37]  Anders P. Eriksson,et al.  Image2Mesh: A Learning Framework for Single Image 3D Reconstruction , 2017, ACCV.

[38]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Hans-Peter Seidel,et al.  Faster isosurface ray tracing using implicit KD-trees , 2005, IEEE Transactions on Visualization and Computer Graphics.

[40]  Anders P. Eriksson,et al.  Implicit Surface Representations As Layers in Neural Networks , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[41]  Pierre-Louis Bazin,et al.  Topology correction of segmented medical images using a fast marching algorithm , 2007, Comput. Methods Programs Biomed..

[42]  Kwangsik Nho,et al.  Associations of the Top 20 Alzheimer Disease Risk Variants With Brain Amyloidosis , 2018, JAMA neurology.

[43]  S. Vos,et al.  Reliability of brain volume measurements: A test-retest dataset , 2014, Scientific Data.

[44]  Qiang Yang,et al.  An Overview of Multi-task Learning , 2018 .

[45]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.