Receptivity to Instructional Computer-Based Feedback (RIF)

Abstract: Feedback can play a pivotal role in learning but is only effective when students use it. However, the use of feedback varies greatly from student to student and these differences must be explained. Hence, the construct receptivity to instructional feedback (RIF) was introduced to assess students’ attitudes towards and engagement with human-provided feedback in multiple English-speaking countries. However, RIF scales have not yet been used in computer-based assessment contexts nor in German-speaking countries; their application is thus limited. This study examined an adapted and translated RIF scale using confirmatory factor analysis (CFA) and exploratory structural equation modeling (ESEM). We capitalized on data sets comprising German-speaking university ( N = 464) and secondary school ( N = 1,207) students. The bifactor-CFA model showed a good model fit and revealed a general receptivity factor alongside the four specific factors: experiential attitudes, instrumental attitudes, cognitive engagement, and behavioral engagement with feedback. Scalar measurement invariance allowed for cross-gender and educational level comparisons. The relationships with broader personality characteristics were consistent with other contexts, thus indicating discriminant validity. General receptivity, cognitive engagement, and experiential attitudes positively correlated with GPA, suggesting convergent validity. Results support the stability of the RIF scale’s adaptation and translation. Assessment implications are discussed.

[1]  D. Iliescu,et al.  Face Validity , 2023, European Journal of Psychological Assessment.

[2]  O. Lüdtke,et al.  Disentangling the Association Between the Big Five Personality Traits and Student Achievement: Meta-Analytic Evidence on the Role of Domain Specificity and Achievement Measures , 2023, Educational Psychology Review.

[3]  D. Schallert,et al.  “Feedback to the future”: Advancing motivational and emotional perspectives in feedback research , 2023, Educational Psychologist.

[4]  Jeffrey K. Smith,et al.  Student – Feedback Interaction Model: Revised , 2022, Studies in Educational Evaluation.

[5]  B. Finn,et al.  Effects of computer-based feedback on lower- and higher-order learning outcomes: A network meta-analysis. , 2022, Journal of Educational Psychology.

[6]  B. Muthén,et al.  Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives. , 2022, Social science research.

[7]  A. Lipnevich,et al.  Receptivity to Instructional Feedback , 2022, European Journal of Psychological Assessment.

[8]  A. Lipnevich,et al.  A Review of Feedback Models and Theories: Descriptions, Definitions, and Conclusions , 2021, Frontiers in Education.

[9]  A. Lipnevich,et al.  A review of feedback typologies and models: Towards an integrative model of feedback elements , 2021, Educational Research Review.

[10]  Jan Retelsdorf,et al.  Assessing value beliefs among university students: Validation of the Value Beliefs Questionnaire for University Students (VBQU) , 2021 .

[11]  Sakhavat Mammadov.  The Big Five Personality Traits and Academic Performance: A Meta-Analysis. , 2021, Journal of personality.

[12]  M. Lauriola,et al.  The Structure of the Emotional Processing Scale (EPS-25) , 2021 .

[13]  Jeffrey K. Smith,et al.  Development of a measure of receptivity to instructional feedback and examination of its links to personality , 2021 .

[14]  David M. Rothschild,et al.  Data quality of platforms and panels for online behavioral research , 2021, Behavior Research Methods.

[15]  H. Marsh,et al.  Value Beliefs About Math , 2020 .

[16]  J. Hattie,et al.  The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research , 2020, Frontiers in Psychology.

[17]  K. Hau,et al.  A Comparison of CFA, ESEM, and BSEM in Test Structure Analysis , 2019, Structural Equation Modeling: A Multidisciplinary Journal.

[18]  Mikael Winberg,et al.  Challenging the Universality of Achievement Goal Models: A Comparison of Two Culturally Distinct Countries , 2018, Scandinavian Journal of Educational Research.

[19]  Erika A. Patall,et al.  A Meta-Analysis of Negative Feedback on Intrinsic Motivation , 2018, Educational Psychology Review.

[20]  Stefan Palan,et al.  Prolific.ac—A subject pool for online experiments , 2017 .

[21]  Tera D. Letzring,et al.  Exploring beyond simple demographic variables: Differences between traditional laboratory samples and crowdsourced online samples on the Big Five personality traits , 2017, Personality and Individual Differences.

[22]  Gilles E. Gignac,et al.  Effect size guidelines for individual differences researchers , 2016 .

[23]  Herbert W. Marsh,et al.  A Bifactor Exploratory Structural Equation Modeling Framework for the Identification of Distinct Sources of Construct-Relevant Psychometric Multidimensionality , 2016 .

[24]  S. Green,et al.  Evaluation of Dimensionality in the Assessment of Internal Consistency Reliability: Coefficient Alpha and Omega Coefficients , 2015 .

[25]  Bengt Muthén,et al.  Bayesian Structural Equation Modeling With Cross-Loadings and Residual Covariances , 2015 .

[26]  V. Savalei,et al.  When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. , 2012, Psychological methods.

[27]  Beth A. Linderbaum,et al.  The Development and Validation of the Feedback Orientation Scale (FOS) , 2010 .

[28]  A. Poropat.  A meta-analysis of the five-factor model of personality and academic performance. , 2009, Psychological bulletin.

[29]  E. Gudmundsson.  Guidelines for translating and adapting psychological instruments , 2009 .

[30]  Phillip W. Braddy,et al.  Power and sensitivity of alternative fit indices in tests of measurement invariance. , 2008, The Journal of applied psychology.

[31]  F. Chen.  Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance , 2007 .

[32]  J. Hattie,et al.  The Power of Feedback , 2007 .

[33]  Beatrice Rammstedt,et al.  Kurzversion des Big Five Inventory (BFI-K): , 2005 .

[34]  Nathan R. Kuncel,et al.  The Validity of Self-Reported Grade Point Averages, Class Ranks, and Test Scores: A Meta-Analysis and Review of the Literature , 2005 .

[35]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[36]  C. Bombardier,et al.  Guidelines for the process of cross-cultural adaptation of self-report measures. , 2000, Spine.

[37]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[38]  J. Block.  A contrarian view of the five-factor approach to personality description. , 1995, Psychological bulletin.

[39]  John S. Hedgcock,et al.  Feedback on feedback: Assessing learner receptivity to teacher response in L2 composing , 1994 .

[40]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[41]  Angelo S. DeNisi,et al.  Item Presentaton as an Influence on Questionnaire Validity: a Field Experiment , 1980 .

[42]  Herbert W. Marsh,et al.  Exploratory structural equation modeling. , 2013 .

[43]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[44]  P. Costa,et al.  Validation of the five-factor model of personality across instruments and observers. , 1987, Journal of personality and social psychology.