RELEX: An Excel-based software tool for sampling split-half reliability coefficients

Abstract Split-half reliability provides a method for estimating inter-item reliability of measures obtained from repeatedly administered items (or trials). We introduce RELEX; a freely available, Microsoft Excel-based software tool that randomly assembles test halves, yielding a sampling-based distribution of split-half reliability coefficients. Estimates of parallel, tau-equivalent, and congeneric split half reliability are reported. RELEX offers a graphical user interface, and provides freely eligible numbers of sampled splits as well as easy handling of missing values and variable trial or item numbers. Results are summarized in the form of a histogram, along with measures of its central tendency and uncertainty.

[1]  Matthijs J. Warrens.  A comparison of reliability coefficients for psychometric tests that consist of two parts , 2016, Adv. Data Anal. Classif..

[2]  J. Rouder,et al.  A psychometrics of individual differences in experimental tasks , 2019, Psychonomic bulletin & review.

[3]  L. S. Feldt.  Estimation of the reliability of a test divided into two parts of unequal length , 1975 .

[4]  C. I. Mosier.  A Short Cut in the Estimation of Split-Halves Coefficients , 1941 .

[5]  J. Grange,et al.  Inhibition in Task Switching: The Reliability of the n − 2 Repetition Cost , 2017, Quarterly journal of experimental psychology.

[6]  C. Spearman.  CORRELATION CALCULATED FROM FAULTY DATA , 1910 .

[7]  W. H. Angoff,et al.  Test reliability and effective test length , 1953 .

[8]  R. Klein,et al.  Appraising the ANT: Psychometric and theoretical considerations of the Attention Network Test. , 2010, Neuropsychology.

[9]  L. Cronbach.  The two disciplines of scientific psychology. , 1957 .

[10]  J. Blechert,et al.  Approach–avoidance tendencies towards food: Measurement on a touchscreen and the role of attention and food craving , 2019, Appetite.

[11]  S. Hofmann,et al.  Attention Bias Modification Training Via Smartphone to Reduce Social Anxiety: A Randomized, Controlled Multi-Session Experiment , 2014, Cognitive Therapy and Research.

[12]  W. Brown.  SOME EXPERIMENTAL RESULTS IN THE CORRELATION OF MENTAL ABILITIES1 , 1910 .

[13]  Shelly R. Cooper,et al.  The Role of Psychometrics in Individual Differences Research in Cognition: A Case Study of the AX-CPT , 2017, Front. Psychol..

[14]  R. Ulrich,et al.  Mental chronometry and individual differences: Modeling reliabilities and correlations of reaction time means and effect sizes , 2013, Psychonomic Bulletin & Review.

[15]  Samuel B. Green,et al.  Use of internal consistency coefficients for estimating reliability of experimental task scores , 2016, Psychonomic bulletin & review.

[16]  Annchen R. Knodt,et al.  The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences , 2017, Behavior Research Methods.

[17]  B. Kopp,et al.  The Reliability of the Wisconsin Card Sorting Test in Clinical Practice , 2019, Assessment.

[18]  J. Jackson Barnette,et al.  ScoreRel CI: An Excel Program for Computing Confidence Intervals for Commonly Used Score Reliability Coefficients , 2005 .

[19]  Louis Guttman,et al.  A basis for analyzing test-retest reliability , 1945, Psychometrika.

[20]  Eunseong Cho.  Making Reliability Reliable , 2016 .

[21]  P. Bentler,et al.  Quantile Lower Bounds to Reliability Based on Locally Optimal Splits , 2013, Psychometrika.

[22]  Evan Mayo-Wilson,et al.  Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. , 2018, The American psychologist.

[23]  J. C. Flanagan.  A proposed procedure for increasing the efficiency of objective tests. , 1937 .

[24]  W Revelle,et al.  Hierarchical Cluster Analysis And The Internal Structure Of Tests. , 1979, Multivariate behavioral research.

[25]  Bruno Kopp,et al.  Data quality over data quantity in computational cognitive neuroscience , 2018, NeuroImage.

[26]  George Houghton,et al.  CDF-XL: computing cumulative distribution functions of reaction time data in Excel , 2011, Behavior research methods.

[27]  E. Fox,et al.  Psychological Science Needs a Standard Practice of Reporting the Reliability of Cognitive-Behavioral Measurements , 2019, Advances in Methods and Practices in Psychological Science.