Bayesian meta‐analysis of coefficient alpha

The current paper describes and illustrates a Bayesian approach to the meta-analysis of coefficient alpha. Alpha is the most commonly used estimate of the reliability or consistency (freedom from measurement error) for educational and psychological measures. The conventional approach to meta-analysis uses inverse variance weights to combine information from independent studies to provide an overall estimate. The Bayesian approach provides similar estimates to the conventional approach if a diffuse prior is used. However, the Bayesian approach also provides 'shrunken' local estimates of reliability in each context. The amount of shrinkage depends upon both the variability in the underlying populations and the sampling variance of the local estimates. Advantages of the approach are the estimation of individual studies adjusted for sampling error and the application of meta-analytic results to new local studies in which the local study 'borrows strength' from the meta-analysis. The ability to borrow strength for the new local studies is particularly useful in applied work in which the estimate of the local parameter is of primary interest. The approach is illustrated by the analysis of studies of the reliability of the General Ethnicity Questionnaire - Abridged, a measure of identification with the culture of one's heritage and the culture of one's host country. Copyright © 2013 John Wiley & Sons, Ltd.

[1]  J. Tsai,et al.  Somatic and Social: Chinese Americans Talk about Emotion , 2004, Personality & social psychology bulletin.

[2]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[3]  Douglas G Bonett,et al.  Meta-analytic interval estimation for bivariate correlations. , 2008, Psychological methods.

[4]  Douglas G. Bonett,et al.  Sample Size Requirements for Testing and Estimating Coefficient Alpha , 2002 .

[5]  T. Hogan,et al.  Reliability Methods: A Note on the Frequency of Use of Various Types , 2000 .

[6]  J. Tsai,et al.  The Meaning of “Being Chinese” and “Being American” , 2000 .

[7]  Sandra L. Wong.  Depression Level in Inner-City Asian American Adolescents , 2000 .

[8]  Douglas G. Bonett,et al.  Varying coefficient meta-analytic methods for alpha reliability. , 2010, Psychological methods.

[9]  Ryan T. Howell,et al.  Reliability of Bidimensional Acculturation Scores , 2009 .

[10]  D. Cella,et al.  Reliability across studies from the functional assessment of cancer therapy-general (FACT-G) and its subscales: a reliability generalization , 2008, Quality of Life Research.

[11]  J. Tsai.  Cultural Orientation of Hmong Young Adults , 2000 .

[12]  M. W. Richardson,et al.  The theory of the estimation of test reliability , 1937 .

[13]  Seungsoo Yeo.  Reliability Generalization of Curriculum-Based Measurement Reading Aloud: A Meta-Analytic Review , 2011 .

[14]  Matt Vassar,et al.  A Reliability Generalization Study of Coefficient Alpha for the UCLA Loneliness Scale , 2008, Journal of personality assessment.

[15]  T. Vacha-Haase,et al.  Reliability Generalization: Exploring Variance in Measurement Error Affecting Score Reliability Across Studies , 1998 .

[16]  J. Tsai,et al.  What does "being American" mean? A comparison of Asian American and European American young adults. , 2002, Cultural diversity & ethnic minority psychology.

[17]  L. Kogan,et al.  Reliability Generalization: Exploring Reliability Variations on MMPI/MMPI-2 Validity Scale Scores , 2001, Assessment.

[18]  S. Hall,et al.  Smoking and Depression in Chinese Americans , 2003, The American journal of the medical sciences.

[19]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[20]  J. Sánchez-Meca,et al.  Some recommended statistical analytic practices when reliability generalization studies are conducted. , 2012, The British journal of mathematical and statistical psychology.

[21]  Michael C. Rodriguez,et al.  Meta-analysis of coefficient alpha. , 2006, Psychological methods.

[22]  Thomas E. Whalen,et al.  A k-sample significance test for independent alpha coefficients , 1976 .

[23]  Siobhan K. O’Toole,et al.  The Ways of Coping Scale , 2008 .

[24]  Roy J. Eidelson,et al.  Revisiting Du Bois: The Relationship Between African American Double Consciousness and Beliefs About Racial and National Group Experiences , 2005 .

[25]  Leland Wilkinson,et al.  Statistical Methods in Psychology Journals Guidelines and Explanations , 2005 .

[26]  Bruce Thompson,et al.  Score Reliability: A Retrospective Look Back at 12 Years of Reliability Generalization Studies , 2011 .

[27]  Leonard S. Feldt,et al.  Variability in Reliability Coefficients and the Standard Error of Measurement From School District to District , 1999 .

[28]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[29]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[30]  Bruce Thompson,et al.  Characterizing Measurement Error in Scores Across Studies: Some Recommendations for Conducting “Reliability Generalization” Studies , 2002 .

[31]  Johanna E. Nilsson,et al.  Practices Regarding Reporting of Reliability Coefficients: A Review of Three Journals , 1999 .

[32]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[33]  Dale Whhtington,et al.  How Well Do Researchers Report their Measures? an Evaluation of Measurementin Published Educational Research , 1998 .

[34]  T. Louis.  Estimating a population of parameter values using Bayes and empirical Bayes methods , 1984 .

[35]  S. Natasha Beretvas,et al.  A Reliability Generalization Study of Scores on Rotter's and Nowicki-Strickland's Locus of Control Scales , 2008 .

[36]  Dawn Iacobucci,et al.  Advancing Alpha: Measuring Reliability with Confidence , 2003 .

[37]  Xitao Fan,et al.  Assessing the Reliability of Beck Depression Inventory Scores: Reliability Generalization across Studies , 2000 .

[38]  Brian Knutson,et al.  Cultural variation in affect valuation. , 2006, Journal of personality and social psychology.