The construct of psychophysiological reactivity: Statistical and psychometric issues

The purpose of this paper is to review major statistical and psychometric issues impacting the study of psychophysiological reactivity and discuss their implications for applied developmental researchers. We first cover traditional approaches such as the observed difference score (DS) and the observed residual score (RS), including a review of classic and recent research on their reliability and validity from two related bodies of work: the measurement of change and the Law of Initial Values. Second, we review several types of latent variable modeling in this context: latent difference score (LDS) models, latent residual score (LRS) models, latent state-trait (LST) models, and latent growth curve (LGC) models. Finally, we provide broad guidelines for applied researchers broken down by key stages of a psychophysiological project: study planning, data analysis, and reporting of results. Our recommendations highlight the need for (1) increased attention to the ubiquitous nature of measurement error in observed variables and the importance of employing latent variable models when possible, and (2) increased specification of theories relating to the construct of reactivity, especially in regards to the distinction between baseline arousal and change over time in broader systems of variables.

[1]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[2]  J. Lacey,et al.  THE EVALUATION OF AUTONOMIC RESPONSES: TOWARD A GENERAL SOLUTION , 1956, Annals of the New York Academy of Sciences.

[3]  J. Jamieson.  Correlates of reactivity: problems with regression based methods. , 1994, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  R. Davidson,et al.  Cardiac reactivity is associated with changes in negative emotion in 24-month-olds. , 2005, Developmental psychobiology.

[5]  C. Harris.  Problems in measuring change , 1965 .

[6]  An Empirical Study of the Relative Error Magnitude in Three Measures of Change. , 1984 .

[7]  K. K. Sharma,et al.  On the Optimum Predictive Potential of Change Measure. , 1988 .

[8]  L. Johnson,et al.  DIFFERENTIAL EFFECT OF THE LAW OF INITIAL VALUE (LIV) ON AUTONOMIC VARIABLES. , 1964, Psychophysiology.

[9]  T. Kamarck.  Recent developments in the study of cardiovascular reactivity: contributions from psychometric theory and social psychology. , 1992, Psychophysiology.

[10]  T. Beauchaine,et al.  Polyvagal Theory and developmental psychopathology: Emotion dysregulation and conduct problems from preschool to adolescence , 2007, Biological Psychology.

[11]  N. Adler,et al.  Biological sensitivity to context: the interactive effects of stress reactivity and family adversity on socioemotional behavior and school readiness. , 2010, Child development.

[12]  D. A. Kenny,et al.  Trait–state models for longitudinal data. , 2001 .

[13]  Phil Wood.  Confirmatory Factor Analysis for Applied Research , 2008 .

[14]  Gregory R. Hancock,et al.  Structural equation modeling : a second course , 2006 .

[15]  J. Furedy,et al.  The law of initial values: differentiated testing as an empirical generalization versus enshrinement as a methodological rule. , 1989, Psychophysiology.

[16]  Julian F. Thayer,et al.  Does resting electroencephalograph asymmetry reflect a trait? an application of latent state-trait theory. , 2002 .

[17]  Andrea K. Webb,et al.  Multilevel models for repeated measures research designs in psychophysiology: an introduction to growth curve modeling. , 2007, Psychophysiology.

[18]  P. D. Oldham,et al.  A note on the analysis of repeated measurements of the same subjects. , 1962, Journal of chronic diseases.

[19]  R. Laird,et al.  The equivalence of regression models using difference scores and models using separate scores for each informant: implications for the study of informant discrepancies. , 2011, Psychological assessment.

[20]  T. Kamarck,et al.  Cardiovascular Reactivity to Psychological Challenge: Conceptual and Measurement Considerations , 2003, Psychosomatic medicine.

[21]  T. Little,et al.  Handbook of personality development , 2014 .

[22]  Large Sample Estimates of the Reliability of Simple, Residualized, and Base-Free Gain Scores , 1987 .

[23]  Tenko Raykov,et al.  Structural models for studying correlates and predictors of change , 1992 .

[24]  L R Bergman,et al.  A person-oriented approach in research on developmental psychopathology , 1997, Development and Psychopathology.

[25]  J. Jamieson.  Measurement of Change and the Law of Initial Values: A Computer Simulation Study , 1995 .

[26]  Alan Booth,et al.  Low salivary cortisol levels and externalizing behavior problems in youth , 2005, Development and Psychopathology.

[27]  J. Rodgers.  The epistemology of mathematical and statistical modeling: a quiet methodological revolution. , 2010, The American psychologist.

[28]  L. Mayes,et al.  The Cambridge Handbook of Environment in Human Development: Contents , 2015 .

[29]  K. Scherer,et al.  Handbook of affective sciences. , 2003 .

[30]  M. El-Sheikh,et al.  Longitudinal relations between marital conflict and child adjustment: vagal regulation as a protective factor. , 2006, Journal of family psychology : JFP : journal of the Division of Family Psychology of the American Psychological Association.

[31]  Jeffrey R. Edwards,et al.  Regression Analysis as an Alternative to Difference Scores , 1994 .

[32]  C. Stein,et al.  Structural equation modeling. , 2012, Methods in molecular biology.

[33]  N. Blomqvist.  On the Relation between Change and Initial Value , 1977 .

[34]  Glenn I. Roisman,et al.  The psychophysiology of adult attachment relationships: Autonomic reactivity in marital and premarital interactions. , 2007, Developmental psychology.

[35]  B. Muthén,et al.  Investigating population heterogeneity with factor mixture models. , 2005, Psychological methods.

[36]  L. Cronbach,et al.  How we should measure "change": Or should we? , 1970 .

[37]  The law of initial values: five factors or two? , 1993, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[38]  K. Quigley,et al.  A comparative validation of sympathetic reactivity in children and adults. , 2006, Psychophysiology.

[39]  W. Thomas Boyce,et al.  Biological Sensitivity to Context , 2008 .

[40]  M. Lewis,et al.  Cortisol response to inoculation in 4-year-old children , 2009, Psychoneuroendocrinology.

[41]  Robert A. Cribbie,et al.  Decreases in posttest variance and the measurement of change , 2004 .

[42]  K. Matthews,et al.  Patterns of sympathetic and parasympathetic reactivity in a sample of children and adolescents. , 2000, Psychophysiology.

[43]  T. Beauchaine,et al.  Vagal tone, development, and Gray's motivational theory: Toward an integrated model of autonomic nervous system functioning in psychopathology , 2001, Development and Psychopathology.

[44]  Michael Eid,et al.  Latent state–trait theory and research in personality and individual differences , 1999 .

[45]  J. Mcardle,et al.  Latent difference score structural models for linear dynamic analyses with incomplete longitudinal data. , 2001 .

[46]  S. Dickerson,et al.  Acute stressors and cortisol responses: a theoretical integration and synthesis of laboratory research. , 2004, Psychological bulletin.

[47]  R. Geenen,et al.  A simple test of the Law of Initial Values. , 1993, Psychophysiology.

[48]  F. Lord.  A paradox in the interpretation of group comparisons. , 1967, Psychological bulletin.

[49]  G. Berntson,et al.  Handbook of Psychophysiology: Cardiovascular Psychophysiology , 2007 .

[50]  David Rogosa,et al.  A growth curve approach to the measurement of change. , 1982 .

[51]  M. Del Giudice,et al.  Adaptive patterns of stress responsivity: a preliminary investigation. , 2012, Developmental psychology.

[52]  Lynda A. King,et al.  Sequential Temporal Dependencies in Associations Between Symptoms of Depression and Posttraumatic Stress Disorder: An Application of Bivariate Latent Difference Score Structural Equation Modeling , 2009, Multivariate behavioral research.

[53]  S. Porges.  The polyvagal theory: phylogenetic substrates of a social nervous system. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[54]  D. Nagin.  Group-based modeling of development , 2005 .

[55]  J. Mcardle.  Latent variable modeling of differences and changes with longitudinal data. , 2009, Annual review of psychology.

[56]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[57]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[58]  D. W. Zimmerman.  The Reliability of Difference Scores in Populations and Samples , 2009 .

[59]  E. Ferrer,et al.  Factorial Invariance and The Specification of Second-Order Latent Growth Models. , 2008, Methodology : European journal of research methods for the behavioral & social sciences.

[60]  J. Overall,et al.  Reassertion of the paradoxical power of tests of significance based on unreliable difference scores. , 1976 .

[61]  Douglas A. Granger,et al.  Asymmetry between salivary cortisol and α-amylase reactivity to stress: Relation to aggressive behavior in adolescents , 2006, Psychoneuroendocrinology.

[62]  S. Porges.  The polyvagal perspective , 2007, Biological Psychology.

[63]  B. Muthén,et al.  How to Use a Monte Carlo Study to Decide on Sample Size and Determine Power , 2002 .

[64]  K. Buss,et al.  Dynamic measures of RSA predict distress and regulation in toddlers. , 2010, Developmental psychobiology.

[65]  D. Wegener,et al.  The problem of equivalent models in applications of covariance structure analysis. , 1993, Psychological bulletin.

[66]  Yu-Kang Tu,et al.  Revisiting the relation between change and initial value: a review and evaluation , 2007, Statistics in medicine.

[67]  Linda M. Collins,et al.  Latent class and latent transition analysis , 2009 .

[68]  R. P. McDonald,et al.  Principles and practice in reporting structural equation analyses. , 2002, Psychological methods.

[69]  Marco Del Giudice,et al.  The Adaptive Calibration Model of stress responsivity , 2011, Neuroscience & Biobehavioral Reviews.

[70]  J. Arthur Woodward,et al.  Unreliability of difference scores: A paradox for measurement of change. , 1975 .

[71]  Jay Belsky,et al.  Beyond diathesis stress: differential susceptibility to environmental influences. , 2009, Psychological bulletin.

[72]  Kristopher J Preacher,et al.  Latent Growth Curve Modeling , 2008 .

[73]  D. W. Zimmerman,et al.  GAIN SCORES IN RESEARCH CAN BE HIGHLY RELIABLE , 1982 .

[74]  J. Edwards.  Ten Difference Score Myths , 2001 .

[75]  D. Cicchetti,et al.  Moving research on resilience into the 21st century: Theoretical and methodological considerations in examining the biological contributors to resilience , 2003, Development and Psychopathology.

[76]  M. V. Dulmen,et al.  Latent state trait modeling of children's cortisol at two points of the diurnal cycle , 2012, Psychoneuroendocrinology.

[77]  Jeffrey D Long,et al.  Measuring Interpersonal Callousness in Boys From Childhood to Adolescence: An Examination of Longitudinal Invariance and Temporal Stability , 2007, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[78]  William J. Gavin,et al.  14 Obtaining Reliable Psychophysiological Data with Child Participants Methodological Considerations , 2007 .

[79]  T. Brown,et al.  Confirmatory Factor Analysis for Applied Research , 2006 .

[80]  Patrick J Curran,et al.  Have Multilevel Models Been Structural Equation Models All Along? , 2003, Multivariate behavioral research.

[81]  H. Heath,et al.  The Quantification of “Response” to Experimental Stimuli , 1965, Psychosomatic medicine.

[82]  J. Cacioppo,et al.  Autonomic determinism: the modes of autonomic control, the doctrine of autonomic space, and the laws of autonomic constraint. , 1991, Psychological review.

[83]  J. Furedy,et al.  Individual differences in phasic cardiac reactivity to psychological stress and the law of initial value. , 1985, Psychophysiology.

[84]  J. Jamieson,et al.  Dealing with baseline differences: two principles and two dilemmas. , 1999, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[85]  James A. Bovaird,et al.  Methods for the Analysis of Change , 2006 .

[86]  N. Schneiderman,et al.  The reliability and specificity of delta versus residualized change as measures of cardiovascular reactivity to behavioral challenges. , 1991, Psychophysiology.

[87]  N. Schneiderman,et al.  Piecewise latent growth curve modeling of systolic blood pressure reactivity and recovery from the cold pressor test. , 2001, Psychophysiology.

[88]  P. Jin,et al.  Toward a reconceptualization of the law of initial value. , 1992, Psychological bulletin.

[89]  Kevin J. Grimm,et al.  Multivariate longitudinal methods for studying developmental relationships between depression and academic achievement , 2007 .

[90]  M. W. Krauss,et al.  Measuring Change in Children and Families , 1991 .

[91]  H. Marsh,et al.  Structural equation models of latent interaction and quadratic effects. , 2013 .

[92]  M. El-Sheikh,et al.  Children’s Externalizing and Internalizing Symptoms over Time: The Role of Individual Differences in Patterns of RSA Responding , 2009, Journal of abnormal child psychology.

[93]  H. Koot,et al.  Cortisol and externalizing behavior in children and adolescents: mixed meta-analytic evidence for the inverse relation of basal cortisol and cortisol reactivity with externalizing behavior. , 2008, Developmental psychobiology.

[94]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[95]  Joel S. Steele,et al.  Longitudinal Data Analysis for the Behavioral Sciences Using R , 2013 .

[96]  S. Segalowitz,et al.  Developmental psychophysiology : theory, systems, and methods , 2007 .

[97]  L. Lengua,et al.  Physiological Profiles During Delay of Gratification: Associations with Emotionality, Self-regulation, and Adjustment Problems. , 2009, Journal of applied developmental psychology.

[98]  V. Neuhaus,et al.  Latent Class Analysis , 2010 .

[99]  Frederic M. Lord,et al.  The Measurement of Growth , 1956 .

[100]  S. Calkins,et al.  Cardiac vagal regulation differentiates among children at risk for behavior problems , 2007, Biological Psychology.

[101]  A. Farrell,et al.  Structural equation modeling with longitudinal data: strategies for examining group differences and reciprocal relationships. , 1994, Journal of consulting and clinical psychology.

[102]  Jelena Obradović,et al.  How can the study of physiological reactivity contribute to our understanding of adversity and resilience processes in development? , 2012, Development and Psychopathology.

[103]  D. Kupfer,et al.  Developmental and contextual influences on autonomic reactivity in young children. , 2003, Developmental psychobiology.

[104]  J. Jamieson,et al.  The law of initial values: a four factor theory. , 1992, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[105]  B. Alpert,et al.  Reliability of cardiovascular reactivity to stress: internal consistency. , 2007, Psychophysiology.

[106]  J. Thayer,et al.  Does resting electroencephalograph asymmetry reflect a trait? an application of latent state-trait theory. , 2002, Journal of personality and social psychology.

[107]  W. Boyce,et al.  Associations Between Physiological Reactivity and Children’s Behavior: Advantages of a Multisystem Approach , 2002, Journal of developmental and behavioral pediatrics : JDBP.

[108]  James A. Bovaird,et al.  On the Merits of Orthogonalizing Powered and Product Terms: Implications for Modeling Interactions Among Latent Variables , 2006 .

[109]  J. Obradović,et al.  Differentiating challenge reactivity from psychomotor activity in studies of children's psychophysiology: considerations for theory and measurement. , 2011, Journal of experimental child psychology.

[110]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[111]  R. MacCallum,et al.  Power analysis and determination of sample size for covariance structure modeling. , 1996 .

[112]  William Meredith,et al.  Latent curve analysis , 1990 .

[113]  S. Sarkar,et al.  The Philosophy of Science: An Encyclopedia , 2006 .

[114]  David Rogosa,et al.  Myths about longitudinal research. , 1988 .

[115]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[116]  S. Toulmin,et al.  Philosophy of Science , 1970, The British Journal for the History of Science.

[117]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[118]  Robert L. Linn,et al.  The Determination of the Significance of Change Between Pre- and Posttesting Periods , 1977 .

[119]  J. Cacioppo,et al.  Inferring psychological significance from physiological signals. , 1990, The American psychologist.

[120]  The effect of prestimulus variability upon physiological reactivity scores. , 1968, Psychophysiology.

[121]  S. Manuck,et al.  Behaviorally-Evoked Cardiovascular Reactivity and Hypertension: Conceptual Issues and Potential Associations , 1990 .

[122]  M. Gunnar,et al.  Identifying atypical cortisol patterns in young children: The benefits of group-based trajectory modeling , 2009, Psychoneuroendocrinology.

[123]  B. Ellis,et al.  Printed in the United States of America DOI: 10.10170S0954579405050145 , 2022 .

[124]  Bruno D. Zumbo,et al.  Reliability of Measurement and Power of Significance Tests Based on Differences , 1993 .

[125]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[126]  E. Shirtcliff,et al.  Gender- and Age-Related Differences in the Association Between Social Relationship Quality and Trait Levels of Salivary Cortisol , 2008 .

[127]  J. Hittner,et al.  Reliability and Validity of Gain Scores Considered Graphically , 2010, Perceptual and motor skills.

[128]  J. Schafer,et al.  Missing data: our view of the state of the art. , 2002, Psychological methods.

[129]  Robert A. Cribbie,et al.  Structural Equation Models and the Regression Bias for Measuring Correlates of Change , 2000 .

[130]  J. Cacioppo,et al.  Origins of baseline variance and the Law of Initial Values. , 1994, Psychophysiology.

[131]  K. Schaie,et al.  Methodological issues in aging research , 1990 .

[132]  J. Willett,et al.  DEMONSTRATING THE RELIABILITY THE DIFFERENCE SCORE IN THE MEASUREMENT OF CHANGE , 1983 .