Neuroimaging of individual differences: A latent variable modeling perspective

HighlightsUnderstanding individual differences requires consideration of psychometricsPsychometric frameworks are not often used in fMRI due to power/sample size issuesLatent variable models are flexible and can address psychometric concernsIndividual differences in fMRI can be better understood via latent variable modelsRecent big data fMRI projects provide opportunity to harness latent variable models ABSTRACT Neuroimaging data is being increasingly utilized to address questions of individual difference. When examined with task‐related fMRI (t‐fMRI), individual differences are typically investigated via correlations between the BOLD activation signal at every voxel and a particular behavioral measure. This can be problematic because: 1) correlational designs require evaluation of t‐fMRI psychometric properties, yet these are not well understood; and 2) bivariate correlations are severely limited in modeling the complexities of brain‐behavior relationships. Analytic tools from psychometric theory such as latent variable modeling (e.g., structural equation modeling) can help simultaneously address both concerns. This review explores the advantages gained from integrating psychometric theory and methods with cognitive neuroscience for the assessment and interpretation of individual differences. The first section provides background on classic and modern psychometric theories and analytics. The second section details current approaches to t‐fMRI individual difference analyses and their psychometric limitations. The last section uses data from the Human Connectome Project to provide illustrative examples of how t‐fMRI individual differences research can benefit by utilizing latent variable models.

[1]  D. Barch,et al.  Introduction to the special issue on reliability and replication in cognitive and affective neuroscience research , 2013, Cognitive, affective & behavioral neuroscience.

[2]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[3]  T. Yarkoni Big Correlations in Little Studies: Inflated fMRI Correlations Reflect Low Statistical Power—Commentary on Vul et al. (2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[4]  A. Boomsma Nonconvergence, improper solutions, and starting values in lisrel maximum likelihood estimation , 1985 .

[5]  David J. Bartholomew,et al.  Latent Variable Models and Factor Analysis: A Unified Approach , 2011 .

[6]  R. Sitgreaves Psychometric theory (2nd ed.). , 1979 .

[7]  B. Biswal,et al.  Functional connectivity of default mode network components: Correlation, anticorrelation, and causality , 2009, Human brain mapping.

[8]  Paul Kline,et al.  A Handbook of Test Construction (Psychology Revivals) : Introduction to Psychometric Design , 2015 .

[9]  Lucina Q. Uddin,et al.  Combining region- and network-level brain-behavior relationships in a structural equation model , 2018, NeuroImage.

[10]  L. Cronbach,et al.  THEORY OF GENERALIZABILITY: A LIBERALIZATION OF RELIABILITY THEORY† , 1963 .

[11]  William D. Marslen-Wilson,et al.  The Cambridge Centre for Ageing and Neuroscience (Cam-CAN) study protocol: a cross-sectional, lifespan, multidisciplinary examination of healthy cognitive ageing , 2014, BMC Neurology.

[12]  Karl J. Friston Ten ironic rules for non-statistical reviewers , 2012, NeuroImage.

[13]  Michael B. Miller,et al.  fMRI reliability: Influences of task and experimental design , 2013, Cognitive, Affective, & Behavioral Neuroscience.

[14]  P. Molenaar,et al.  Determining the number of factors in P-technique factor analysis , 2017 .

[15]  Essa Yacoub,et al.  The WU-Minn Human Connectome Project: An overview , 2013, NeuroImage.

[16]  Denes Szucs,et al.  Correction: Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature , 2016, bioRxiv.

[17]  P. Elliott,et al.  UK Biobank: An Open Access Resource for Identifying the Causes of a Wide Range of Complex Diseases of Middle and Old Age , 2015, PLoS medicine.

[18]  D. Borsboom,et al.  Simpson's paradox in psychological science: a practical guide , 2013, Front. Psychol..

[19]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[20]  Karl J. Friston,et al.  Statistical parametric mapping , 2013 .

[21]  D. Borsboom,et al.  The Theoretical Status of Latent Variables , 2003 .

[22]  Scott T. Grafton,et al.  Extensive Individual Differences in Brain Activations Associated with Episodic Retrieval are Reliable Over Time , 2002, Journal of Cognitive Neuroscience.

[23]  Michael B. Miller,et al.  How sample size influences the replicability of task-based fMRI , 2017, bioRxiv.

[24]  R. O’Reilly The What and How of prefrontal cortical organization , 2010, Trends in Neurosciences.

[25]  Anthony Randal McIntosh,et al.  Partial Least Squares (PLS) methods for neuroimaging: A tutorial and review , 2011, NeuroImage.

[26]  Michael B. Miller,et al.  How reliable are the results from functional magnetic resonance imaging? , 2010, Annals of the New York Academy of Sciences.

[27]  Kaylie A. Carbine,et al.  Sample size calculations in human electrophysiology (EEG and ERP) studies: A systematic review and recommendations for increased rigor. , 2017, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[28]  Olaf Sporns,et al.  Using confirmatory factor analysis to measure contemporaneous activation of defined neuronal networks in functional magnetic resonance imaging , 2012, NeuroImage.

[29]  T. Little Longitudinal Structural Equation Modeling , 2013 .

[30]  R. Engle,et al.  The generality of working memory capacity: a latent-variable approach to verbal and visuospatial memory span and reasoning. , 2004, Journal of experimental psychology. General.

[31]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[32]  D. Borsboom,et al.  The two disciplines of scientific psychology, or: the disunity of psychology as a working hypothesis , 2009 .

[33]  Peter C M Molenaar,et al.  Greater BOLD activity but more efficient connectivity is associated with better cognitive performance within a sample of nicotine‐deprived smokers , 2014, Addiction biology.

[34]  D. Tank,et al.  Brain magnetic resonance imaging with contrast dependent on blood oxygenation. , 1990, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Krzysztof J. Gorgolewski,et al.  Making big data open: data sharing in neuroimaging , 2014, Nature Neuroscience.

[36]  Denny Borsboom,et al.  The attack of the psychometricians , 2006, Psychometrika.

[37]  T. Braver,et al.  Vive les differences! Individual variation in neural mechanisms of executive control , 2010, Current Opinion in Neurobiology.

[38]  Andreas Ritter,et al.  Structural Equations With Latent Variables , 2016 .

[39]  Daniel H. Mathalon,et al.  Using Brain Imaging Measures in Studies of Procognitive Pharmacologic Agents in Schizophrenia: Psychometric and Quality Assurance Considerations , 2011, Biological Psychiatry.

[40]  Jonathan D. Power,et al.  Intrinsic and Task-Evoked Network Architectures of the Human Brain , 2014, Neuron.

[41]  David J. Therriault,et al.  A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence , 2002 .

[42]  Thomas E. Nichols,et al.  Scanning the horizon: towards transparent and reproducible neuroimaging research , 2016, Nature Reviews Neuroscience.

[43]  H. Chernoff,et al.  Why significant variables aren’t automatically good predictors , 2015, Proceedings of the National Academy of Sciences.

[44]  J. Shao AN ASYMPTOTIC THEORY FOR LINEAR MODEL SELECTION , 1997 .

[45]  D. Mook,et al.  In defense of external invalidity. , 1983 .

[46]  Max C. Keuken,et al.  The impact of MRI scanner environment on perceptual decision-making , 2015, Behavior Research Methods.

[47]  J. Allman,et al.  Mapping human visual cortex with positron emission tomography , 1986, Nature.

[48]  R. Gur,et al.  Development of Abbreviated Nine-Item Forms of the Raven’s Standard Progressive Matrices Test , 2012, Assessment.

[49]  Abraham Z. Snyder,et al.  Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion , 2012, NeuroImage.

[50]  Peter Herscovitch,et al.  Tonotopic organization in human auditory cortex revealed by positron emission tomography , 1985, Hearing Research.

[51]  K. Bollen Model Implied Instrumental Variables (MIIVs): An Alternative Orientation to Structural Equation Modeling , 2018, Multivariate behavioral research.

[52]  Elisabeth J. Ploran,et al.  Distinct Stages of Moment‐to‐Moment Processing in the Cinguloopercular and Frontoparietal Networks , 2017, Cerebral cortex.

[53]  Leslie G. Ungerleider,et al.  Changes in limbic and prefrontal functional interactions in a working memory task for faces. , 1996, Cerebral cortex.

[54]  Kathleen M. Gates,et al.  Group search algorithm recovers effective connectivity maps for individuals in homogeneous and heterogeneous samples , 2012, NeuroImage.

[55]  Timothy O. Laumann,et al.  Functional Network Organization of the Human Brain , 2011, Neuron.

[56]  Wm. R. Wright General Intelligence, Objectively Determined and Measured. , 1905 .

[57]  W. K. Simmons,et al.  Circular analysis in systems neuroscience: the dangers of double dipping , 2009, Nature Neuroscience.

[58]  D. A. Kenny,et al.  Correlation and Causation. , 1982 .

[59]  R. Adolphs,et al.  Building a Science of Individual Differences from fMRI , 2016, Trends in Cognitive Sciences.

[60]  R. Schlösser,et al.  Assessing the working memory network: Studies with functional magnetic resonance imaging and structural equation modeling , 2006, Neuroscience.

[61]  John D Medaglia,et al.  Examining working memory task acquisition in a disrupted neural network. , 2011, Brain : a journal of neurology.

[62]  A. Gevins,et al.  Spatiotemporal dynamics of component processes in human working memory. , 1993, Electroencephalography and clinical neurophysiology.

[63]  CM Bennett,et al.  Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: an argument for multiple comparisons correction , 2009, NeuroImage.

[64]  A. Tomarken,et al.  Structural equation modeling: strengths, limitations, and misconceptions. , 2005, Annual review of clinical psychology.

[65]  Timothy O. Laumann,et al.  Generation and Evaluation of a Cortical Area Parcellation from Resting-State Correlations. , 2016, Cerebral cortex.

[66]  D. A. Kenny,et al.  Correlation and Causation , 1937, Wilmott.

[67]  N. Logothetis What we can do and what we cannot do with fMRI , 2008, Nature.

[68]  Denny Meyer,et al.  Structural equation modeling in psychology: the history, development and current challenges , 2014 .

[69]  Kathleen M. Gates,et al.  Extended unified SEM approach for modeling event-related fMRI data , 2011, NeuroImage.

[70]  Thomas E. Nichols,et al.  Statistical Challenges in “Big Data” Human Neuroimaging , 2018, Neuron.

[71]  E. H. Simpson,et al.  The Interpretation of Interaction in Contingency Tables , 1951 .

[72]  Jonathan D. Power,et al.  Statistical improvements in functional magnetic resonance imaging analyses produced by censoring high‐motion data points , 2014, Human brain mapping.

[73]  T. Braver,et al.  Cognitive Neuroscience Approaches to Individual Differences in Working Memory and Executive Control: Conceptual and Methodological Issues , 2010 .

[74]  Cameron S Carter,et al.  A convergent-divergent approach to context processing, general intellectual functioning, and the genetic liability to schizophrenia. , 2005, Neuropsychology.

[75]  P. Meehl Factors and Taxa, Traits and Types, Differences of Degree and Differences in Kind , 1992 .

[76]  Martin A. Lindquist,et al.  Ironing out the statistical wrinkles in “ten ironic rules” , 2013, NeuroImage.

[77]  L. Cronbach The two disciplines of scientific psychology. , 1957 .

[78]  R. MacCallum,et al.  Model modifications in covariance structure analysis: the problem of capitalization on chance. , 1992, Psychological bulletin.

[79]  N. Fox,et al.  NIH Toolbox for Assessment of Neurological and Behavioral Function , 2013, Neurology.

[80]  H. Pashler,et al.  Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 1 , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[81]  Karl J. Friston,et al.  Generalisability, Random Effects & Population Inference , 1998, NeuroImage.

[82]  Rex E. Jung,et al.  Personality and complex brain networks: The role of openness to experience in default network efficiency , 2015, Human brain mapping.

[83]  Mark Jenkinson,et al.  The minimal preprocessing pipelines for the Human Connectome Project , 2013, NeuroImage.

[84]  Steven C. R. Williams,et al.  Measuring fMRI reliability with the intra-class correlation coefficient , 2009, NeuroImage.

[85]  Vince D. Calhoun,et al.  Multimodal neural correlates of cognitive control in the Human Connectome Project , 2017 .

[86]  Shelly R. Cooper,et al.  The Role of Psychometrics in Individual Differences Research in Cognition: A Case Study of the AX-CPT , 2017, Front. Psychol..

[87]  Stefano Palminteri,et al.  When are inter-individual brain-behavior correlations informative? , 2016, bioRxiv.

[88]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[89]  T. Braver The variable nature of cognitive control: a dual mechanisms framework , 2012, Trends in Cognitive Sciences.

[90]  Edward E. Smith,et al.  A Parametric Study of Prefrontal Cortex Involvement in Human Working Memory , 1996, NeuroImage.

[91]  Abraham Z. Snyder,et al.  Function in the human connectome: Task-fMRI and individual differences in behavior , 2013, NeuroImage.

[92]  T. Yarkoni,et al.  Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning , 2017, Perspectives on psychological science : a journal of the Association for Psychological Science.

[93]  Kenneth A. Bollen,et al.  STRUCTURAL EQUATION MODELS THAT ARE NONLINEAR IN LATENT VARIABLES: A LEAST- SQUARES ESTIMATOR , 1995 .

[94]  Randall W Engle,et al.  Working memory, short-term memory, and general fluid intelligence: a latent-variable approach. , 1999, Journal of experimental psychology. General.

[95]  Kenneth A. Bollen,et al.  An alternative two stage least squares (2SLS) estimator for latent variable equations , 1996 .

[96]  M. Miller,et al.  Sample Size Requirements for Structural Equation Models , 2013, Educational and psychological measurement.

[97]  Michael W. Cole,et al.  Global Connectivity of Prefrontal Cortex Predicts Cognitive Control and Intelligence , 2012, The Journal of Neuroscience.

[98]  M. Stone An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[99]  Wei Zhu,et al.  Unified structural equation modeling approach for the analysis of multisubject, multivariate functional MRI data , 2007, Human brain mapping.

[100]  Thomas E. Nichols,et al.  Fixing the stimulus-as-fixed-effect fallacy in task fMRI , 2016, bioRxiv.

[101]  Jonathan D. Power,et al.  Studying Brain Organization via Spontaneous fMRI Signal , 2014, Neuron.

[102]  Sacha Epskamp,et al.  semPlot: Unified Visualizations of Structural Equation Models , 2015 .

[103]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[104]  Evan M. Gordon,et al.  Evidence for Two Independent Factors that Modify Brain Networks to Meet Task Goals. , 2016, Cell reports.

[105]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[106]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[107]  Evan M. Gordon,et al.  Functional System and Areal Organization of a Highly Sampled Individual Human Brain , 2015, Neuron.

[108]  Jesper Andersson,et al.  A multi-modal parcellation of human cerebral cortex , 2016, Nature.

[109]  R. C. Fraley,et al.  Is depression best viewed as a continuum or discrete category? A taxometric analysis of childhood and adolescent depression in a population-based sample. , 2005, Journal of abnormal psychology.

[110]  Turhan Canli,et al.  Variance maps as a novel tool for localizing regions of interest in imaging studies of individual differences , 2005, Cognitive, affective & behavioral neuroscience.

[111]  R. Buckner,et al.  Functional Specialization in the Human Brain Estimated By Intrinsic Hemispheric Interaction , 2014, The Journal of Neuroscience.