The Comparison of the Equated Tests Scores by Using Various Covariates using Bayesian Nonparametric Model

This research is based on obtaining equated scores by using covariates in the Bayesian nonparametric model. As covariates in the study, gender, mathematics self-efficacy scores, and common item scores were used. The distributions were obtained for all score groups. Hellinger Distance was calculated to obtain the distances between the distributions of equated scores by using covariates and the distribution of the target test scores. These distances were compared with the distributions of equated scores obtained from methods based on Item Response Theory. The study was conducted on Canadian and Italian samples of Programme for International Student Assessment (PISA) 2012. PARSCALE and IRTEQ were used for classical methods, and R was used for Bayesian nonparametric model. When gender, mathematics self-efficacy scores, and common item scores were used as covariates in the model, distance values of obtained equated scores to target test scores were close to each other, but their distributions were different. The closest distribution to target test scores was achieved when gender and mathematics self-efficacy scores were used together as covariates in the model, and the farthest distributions were obtained from item response theory methods. As a result of the research, it was determined that the model is more informative than the classical methods.

[1]  M. Wiberg.  A note on equating test scores with covariates , 2015 .

[2]  Fernando A. Quintana,et al.  On the Support of MacEachern’s Dependent Dirichlet Processes and Extensions , 2012 .

[3]  P. Cheng,et al.  Estimating Comparable Scores Using Surrogate Variables , 2001 .

[4]  Jason R. W. Merrick,et al.  A Hellinger distance approach to MCMC diagnostics , 2014 .

[5]  Jeffrey N. Rouder,et al.  Default Bayes factors for ANOVA designs , 2012 .

[6]  Samuel A. Livingston,et al.  Collateral Information for Equating in Small Samples: A Preliminary Investigation , 2011 .

[7]  Fernando A. Quintana,et al.  A Dependent Bayesian Nonparametric Model for Test Equating , 2015 .

[8]  Jorge González,et al.  Bayesian nonparametric estimation of test equating functions with covariates , 2015, Comput. Stat. Data Anal..

[9]  Thomas R. Boucher,et al.  Test Equating, Scaling, and Linking: Methods and Practices , 2007 .

[10]  Lei Mee Thien,et al.  Factors Associated with Malaysian Mathematics Performance in PISA 2012 , 2016 .

[11]  S. MacEachern,et al.  An ANOVA Model for Dependent Random Measures , 2004 .

[12]  N. Betz,et al.  An Exploration of the Mathematics Self-Efficacy/Mathematics Performance Correspondence , 1989 .

[13]  Fernando A. Quintana,et al.  Fully Nonparametric Regression for Bounded Data Using Dependent Bernstein Polynomials , 2017 .

[14]  M. Wiberg,et al.  Nonequivalent groups with covariates design using propensity scores for kernel equating , 2016 .

[15]  Sonia Petrone.  Random Bernstein Polynomials , 1999 .

[16]  M. Liou.  ESTABLISHING SCORE COMPARABILITY IN HETEROGENEOUS POPULATIONS , 1998 .

[17]  A. V. Davier,et al.  The Accuracy and Consistency of a Series of IRT True Score Equatings , 2012 .

[18]  D. Betsy McCoach,et al.  Increasing Student Mathematics Self-Efficacy Through Teacher Training , 2007 .

[19]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[20]  Franz J. Neyer,et al.  A Gentle Introduction to Bayesian Analysis: Applications to Developmental Research , 2013, Child development.

[21]  Yi Ding.  How Do Students’ Mathematics Self-Efficacy, Mathematics Self-Concept and Mathematics Anxiety Influence Mathematical Literacy? — A Comparison between Shanghai-China and Sweden in PISA 2012 , 2016 .

[22]  Yee Whye Teh,et al.  Bayesian Nonparametric Models , 2010, Encyclopedia of Machine Learning.

[23]  Zoubin Ghahramani,et al.  Determinantal clustering process - a nonparametric Bayesian approach to kernel based semi-supervised clustering , 2013, UAI 2013.

[24]  M. J. Kolen.  Traditional Equating Methodology , 1988 .

[25]  E. Boone,et al.  A frequentist assessment of Bayesian inclusion probabilities for screening predictors , 2011 .

[26]  SMALL‐SAMPLE EQUATING WITH PRIOR INFORMATION , 2009 .

[27]  Sooyeon Kim,et al.  Effectiveness of Collateral Information for Improving Equating in Small Samples , 2009 .

[28]  Marie Wiberg,et al.  Observed Score Linear Equating with Covariates , 2011 .

[29]  J. Kruschke.  Doing Bayesian Data Analysis , 2010 .

[30]  M. Linn,et al.  New trends in gender and mathematics performance: a meta-analysis. , 2010, Psychological bulletin.

[31]  Fernando A. Quintana,et al.  Nonparametric Bayesian data analysis , 2004 .

[32]  P. Holland,et al.  POPULATION INVARIANCE AND THE EQUATABILITY OF TESTS: BASIC THEORY AND THE LINEAR CASE , 2000 .

[33]  Marie Wiberg,et al.  Statistical Assessment of Estimated Transformations in Observed-Score Equating. , 2016 .

[34]  T. L. Kelley.  The selection of upper and lower groups for the validation of test items. , 1939 .

[35]  C. Hall,et al.  Gender differences in mathematical performance , 1988 .

[36]  Wolfram Schulz.  Mathematics Self-Efficacy and Student Expectations. Results from PISA 2003. , 2005 .

[37]  Alina A. von Davier,et al.  Examining the Impact of Covariates on Anchor Tests to Ascertain Quality Over Time in a College Admissions Test , 2017 .

[38]  Aremu Ayotola,et al.  The relationship between mathematics self-efficacy and achievement in mathematics☆ , 2009 .

[39]  Marie Wiberg,et al.  Kernel Equating Under the Non-Equivalent Groups With Covariates Design , 2015, Applied psychological measurement.

[40]  P. Holland,et al.  CHOICE OF ANCHOR TEST IN EQUATING , 2006 .

[41]  Sonia Petrone.  Bayesian density estimation using bernstein polynomials , 1999 .

[42]  Stephen G. Walker,et al.  A Bayesian Nonparametric Approach to Test Equating , 2009 .

[43]  Shameem Nyla.  NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION , 2004 .

[44]  PISA 2012 Matematik Okuryazarlığını Etkileyen Faktörlerin Aracılık Modeli ile İncelenmesi , 2015 .

[45]  T. Louis,et al.  Unified frequentist and Bayesian testing of a precise hypothesis , 1997 .