A comparison of automated segmentation and manual tracing in estimating hippocampal volume in ischemic stroke and healthy control participants

Manual quantification of the hippocampal atrophy state and rate is time consuming and prone to poor reproducibility, even when performed by neuroanatomical experts. The automation of hippocampal segmentation has been investigated in normal aging, epilepsy, and in Alzheimer's disease. Our first goal was to compare manual and automated hippocampal segmentation in ischemic stroke and to, secondly, study the impact of stroke lesion presence on hippocampal volume estimation. We used eight automated methods to segment T1-weighted MR images from 105 ischemic stroke patients and 39 age-matched controls sampled from the Cognition And Neocortical Volume After Stroke (CANVAS) study. The methods were: AdaBoost, Atlas-based Hippocampal Segmentation (ABHS) from the IDeALab, Computational Anatomy Toolbox (CAT) using 3 atlas variants (Hammers, LPBA40 and Neuromorphometics), FIRST, FreeSurfer v5.3, and FreeSurfer v6.0-Subfields. A number of these methods were employed to re-segment the T1 images for the stroke group after the stroke lesions were masked (i.e., removed). The automated methods were assessed on eight measures: process yield (i.e. segmentation success rate), correlation (Pearson's R and Shrout's ICC), concordance (Lin's RC and Kandall's W), slope ‘a’ of best-fit line from correlation plots, percentage of outliers from Bland-Altman plots, and significance of control−stroke difference. We eliminated the redundant measures after analysing between-measure correlations using Spearman's rank correlation. We ranked the automated methods based on the sum of the remaining non-redundant measures where each measure ranged between 0 and 1. Subfields attained an overall score of 96.3%, followed by AdaBoost (95.0%) and FIRST (94.7%). CAT using the LPBA40 atlas inflated hippocampal volumes the most, while the Hammers atlas returned the smallest volumes overall. FIRST (p = 0.014), FreeSurfer v5.3 (p = 0.007), manual tracing (p = 0.049), and CAT using the Neuromorphometics atlas (p = 0.017) all showed a significantly reduced hippocampal volume mean for the stroke group compared to control at three months. Moreover, masking of the stroke lesions prior to segmentation resulted in hippocampal volumes which agreed less with manual tracing. These findings recommend an automated segmentation without lesion masking as a more reliable procedure for the estimation of hippocampal volume in ischemic stroke.

[1]  H. Möller,et al.  Differences in hippocampal volume between major depression and schizophrenia: a comparative neuroimaging study , 2010, European Archives of Psychiatry and Clinical Neuroscience.

[2]  M W Weiner,et al.  Hippocampal and cortical atrophy predict dementia in subcortical ischemic vascular disease , 2000, Neurology.

[3]  Karl J. Friston,et al.  Unified segmentation , 2005, NeuroImage.

[4]  G. Pell,et al.  Hippocampal volume assessment in temporal lobe epilepsy: How good is automated segmentation? , 2009, Epilepsia.

[5]  Hans-Jürgen Möller,et al.  Reduced hippocampal volume correlates with executive dysfunctioning in major depression. , 2006, Journal of psychiatry & neuroscience : JPN.

[6]  A. M. Dale,et al.  A hybrid approach to the skull stripping problem in MRI , 2004, NeuroImage.

[7]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[8]  Shen Fei,et al.  Poststroke Dementia , 2005 .

[9]  Jagath C. Rajapakse,et al.  Statistical approach to segmentation of single-channel cerebral MR images , 1997, IEEE Transactions on Medical Imaging.

[10]  Sudha Seshadri,et al.  Lifetime risk of stroke and dementia: current concepts, and estimates from the Framingham Study , 2007, The Lancet Neurology.

[11]  Wim Fias,et al.  Brain networks under attack: robustness properties and the impact of lesions. , 2016, Brain : a journal of neurology.

[12]  Anders M. Dale,et al.  Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex , 2001, IEEE Transactions on Medical Imaging.

[13]  D. Hedges,et al.  Hippocampal volume deficits associated with exposure to psychological trauma and posttraumatic stress disorder in adults: A meta-analysis , 2010, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[14]  Bruce Fischl,et al.  Within-subject template estimation for unbiased longitudinal image analysis , 2012, NeuroImage.

[15]  P. Wiggermann,et al.  Interobserver Agreement in MR Enterography for Diagnostic Assessment in Patients with Crohn's Disease. , 2013, RoFo : Fortschritte auf dem Gebiete der Rontgenstrahlen und der Nuklearmedizin.

[16]  Liana G. Apostolova,et al.  Comparison of AdaBoost and Support Vector Machines for Detecting Alzheimer's Disease Through Automated Hippocampal Segmentation , 2010, IEEE Transactions on Medical Imaging.

[17]  Daniel Rueckert,et al.  Automatic anatomical brain MRI segmentation combining label propagation and decision fusion , 2006, NeuroImage.

[18]  David Manset,et al.  Reproducibility of hippocampal atrophy rates measured with manual, FreeSurfer, AdaBoost, FSL/FIRST and the MAPS-HBSI methods in Alzheimer's disease , 2016, Psychiatry Research: Neuroimaging.

[19]  Arthur W. Toga,et al.  Construction of a 3D probabilistic atlas of human cortical structures , 2008, NeuroImage.

[20]  Anders M. Dale,et al.  Sequence-independent segmentation of magnetic resonance images , 2004, NeuroImage.

[21]  Christian Gaser,et al.  CAT-A Computational Anatomy Toolbox for the Analysis of Structural MRI Data , 2016 .

[22]  Daniel Rueckert,et al.  Multi-atlas based segmentation of brain images: Atlas selection and its effect on accuracy , 2009, NeuroImage.

[23]  S. Roper,et al.  The hippocampus: detailed assessment of normative two-dimensional measurements, signal intensity, and subfield conspicuity on routine 3T T2-weighted sequences , 2017, Surgical and Radiologic Anatomy.

[24]  H. Ko,et al.  Early-onset and delayed-onset poststroke dementia — revisiting the mechanisms , 2017, Nature Reviews Neurology.

[25]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[26]  M. Kendall,et al.  The Problem of $m$ Rankings , 1939 .

[27]  D. Altman,et al.  Measuring agreement in method comparison studies , 1999, Statistical methods in medical research.

[28]  P. Scheltens,et al.  Atrophy of medial temporal lobes on MRI in "probable" Alzheimer's disease and normal ageing: diagnostic value and neuropsychological correlates. , 1992, Journal of neurology, neurosurgery, and psychiatry.

[29]  G. Jackson,et al.  Charting Cognitive and Volumetric Trajectories after Stroke: Protocol for the Cognition and Neocortical Volume after Stroke (CANVAS) Study , 2014, International journal of stroke : official journal of the International Stroke Society.

[30]  Norbert Schuff,et al.  Hippocampal Volume Differences in Gulf War Veterans with Current Versus Lifetime Posttraumatic Stress Disorder Symptoms , 2011, Biological Psychiatry.

[31]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[32]  A M Dale,et al.  Measuring the thickness of the human cerebral cortex from magnetic resonance images. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Nils Daniel Forkert,et al.  Classifiers for Ischemic Stroke Lesion Segmentation: A Comparison Study , 2015, PloS one.

[34]  Xavier Lladó,et al.  Evaluating the effect of multiple sclerosis lesions on automatic brain structure segmentation , 2017, NeuroImage: Clinical.

[35]  D. Louis Collins,et al.  The EADC-ADNI Harmonized Protocol for manual hippocampal segmentation on magnetic resonance: Evidence of validity , 2015, Alzheimer's & Dementia.

[36]  D. Leys,et al.  Poststroke dementia. , 2006, Cerebrovascular diseases.

[37]  Michael Weiner,et al.  and the Alzheimer’s Disease Neuroimaging Initiative* , 2007 .

[38]  Paul A. Yushkevich,et al.  Multi-Atlas Segmentation with Joint Label Fusion , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Liana G. Apostolova,et al.  Delphi definition of the EADC-ADNI Harmonized Protocol for hippocampal segmentation on magnetic resonance , 2015, Alzheimer's & Dementia.

[40]  Anders M. Dale,et al.  Reliability of MRI-derived measurements of human cerebral cortical thickness: The effects of field strength, scanner upgrade and manufacturer , 2006, NeuroImage.

[41]  N. Makris,et al.  MRI-based brain volumetrics: emergence of a developmental brain science , 1999, Brain and Development.

[42]  J. Bremner,et al.  MR-based in vivo hippocampal volumetrics: 2. Findings in neuropsychiatric disorders , 2005, Molecular Psychiatry.

[43]  Boris C. Bernhardt,et al.  Automatic hippocampal segmentation in temporal lobe epilepsy: Impact of developmental abnormalities , 2012, NeuroImage.

[44]  Bruce Fischl,et al.  Highly accurate inverse consistent registration: A robust approach , 2010, NeuroImage.

[45]  N C Andreasen,et al.  A new method for the in vivo volumetric measurement of the human hippocampus with high neuroanatomical accuracy , 2000, Hippocampus.

[46]  Sabina Sonia Tangaro,et al.  Automated hippocampal segmentation in 3D MRI using random undersampling with boosting algorithm , 2015, Pattern Analysis and Applications.

[47]  R M Peshock,et al.  MR Imaging of Hippocampal Asymmetry at 3T in a Multiethnic, Population-Based Sample: Results from the Dallas Heart Study , 2013, American Journal of Neuroradiology.

[48]  Milan Sonka,et al.  3D Slicer as an image computing platform for the Quantitative Imaging Network. , 2012, Magnetic resonance imaging.

[49]  David Manset,et al.  Brain investigation and brain conceptualization. , 2013, Functional neurology.

[50]  Martin Styner,et al.  A comparison of automated segmentation and manual tracing for quantifying hippocampal and amygdala volumes , 2009, NeuroImage.

[51]  Dafna Ben Bashat,et al.  Cognitive Decline After Stroke: Relation to Inflammatory Biomarkers and Hippocampal Volume , 2013, Stroke.

[52]  P. Sprent,et al.  Applied statistics: analysis of variance and regression , 1975 .

[53]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[54]  C. Mathers,et al.  Preventing stroke: saving lives around the world , 2007, The Lancet Neurology.

[55]  F. Massey.  The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[56]  A. Khundakar,et al.  Hippocampal Neuronal Atrophy and Cognitive Function in Delayed Poststroke and Aging-Related Dementias , 2012, Stroke.

[57]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[58]  D. Giavarina.  Understanding Bland Altman analysis , 2015, Biochemia medica.

[59]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[60]  Satrajit S. Ghosh,et al.  Mindboggle: Automated brain labeling with multiple atlases , 2005, BMC Medical Imaging.

[61]  Koenraad Van Leemput,et al.  A computational atlas of the hippocampal formation using ex vivo, ultra-high resolution MRI: Application to adaptive segmentation of in vivo MRI , 2015, NeuroImage.

[62]  Bruce Fischl,et al.  Geometrically Accurate Topology-Correction of Cortical Surfaces Using Nonseparating Loops , 2007, IEEE Transactions on Medical Imaging.

[63]  Daniel Rueckert,et al.  Automatic segmentation of brain MRIs of 2-year-olds into 83 regions of interest , 2008, NeuroImage.

[64]  Alan C. Evans,et al.  A nonparametric method for automatic correction of intensity nonuniformity in MRI data , 1998, IEEE Transactions on Medical Imaging.

[65]  Alan C. Evans,et al.  Fast and robust parameter estimation for statistical partial volume models in brain MRI , 2004, NeuroImage.

[66]  Stephen M. Smith,et al.  A Bayesian model of shape and appearance for subcortical brain segmentation , 2011, NeuroImage.

[67]  T. Mareci,et al.  A majority rule approach for region-of-interest-guided streamline fiber tractography , 2015, Brain Imaging and Behavior.

[68]  L. Whalley,et al.  A comparison of measurement methods of hippocampal atrophy rate for predicting Alzheimer's dementia in the Aberdeen Birth Cohort of 1936 , 2016, Alzheimer's & dementia.

[69]  Nikos Makris,et al.  Automatically parcellating the human cerebral cortex. , 2004, Cerebral cortex.

[70]  Raid Amin,et al.  Applied Statistics: Analysis of Variance and Regression , 2004, Technometrics.

[71]  G. Jackson,et al.  Structural MRI markers of brain aging early after ischemic stroke , 2017, Neurology.

[72]  Frederik Barkhof,et al.  Hippocampal volume change measurement: Quantitative assessment of the reproducibility of expert manual outlining and the automated methods FreeSurfer and FIRST , 2014, NeuroImage.