Comparing the reliability and predictive power of child, teacher, and guardian reports of noncognitive skills

Significance Recent evidence has shown that noncognitive or socioemotional skills (e.g., persistence and self-control) are predictive of success in life and can be shaped through interventions. Accordingly, policy makers and researchers have increasingly measured children’s noncognitive skills, typically relying on surveys in which children rate their own skills or adults rate the skills of children. Such ratings are often collected from multiple respondent types. This study demonstrates that, compared with child and guardian reports of noncognitive skills, teacher reports are more predictive of children’s cognitive and behavioral outcomes in school. Child and guardian reports add minimal predictive power beyond teacher reports. These findings suggest that policy makers and researchers should prioritize teacher reports above those of children and guardians. Children’s noncognitive or socioemotional skills (e.g., persistence and self-control) are typically measured using surveys in which either children rate their own skills or adults rate the skills of children. For many purposes—including program evaluation and monitoring school systems—ratings are often collected from multiple perspectives about a single child (e.g., from both the child and an adult). Collecting data from multiple perspectives is costly, and there is limited evidence on the benefits of this approach. Using a longitudinal survey, this study compares children’s noncognitive skills as reported by themselves, their guardians, and their teachers. Although reports from all three types of respondents are correlated with each other, teacher reports have the highest internal consistency and are the most predictive of children’s later cognitive outcomes and behavior in school. The teacher reports add predictive power beyond baseline measures of Intelligence Quotient (IQ) for most outcomes in schools. Measures collected from children and guardians add minimal predictive power beyond the teacher reports.

[1]  Dirk T. Tempelaar,et al.  Personality traits and academic performance: Correcting self-assessed traits with vignettes , 2021, PloS one.

[2]  Beatrice Rammstedt,et al.  Big Five Inventory , 2012, Encyclopedia of Personality and Individual Differences.

[3]  J. Heckman,et al.  Sensitivity of self-reported noncognitive skills to survey administration conditions , 2019, Proceedings of the National Academy of Sciences.

[4]  Sidney K. D'Mello,et al.  Why High School Grades Are Better Predictors of On-Time College Graduation Than Are Admissions Test Scores: The Roles of Self-Regulation and Cognitive Ability , 2019, American Educational Research Journal.

[5]  Jonah E. Rockoff,et al.  What Do Test Scores Miss? The Importance of Teacher Effects on Non–Test Score Outcomes , 2018, Journal of Political Economy.

[6]  L. Stankov,et al.  The Effects of Vignette Scoring on Reliability and Validity of Self-Reports , 2018, Applied psychological measurement.

[7]  R. Roberts,et al.  Using Anchoring Vignettes to Adjust Self-Reported Personality: A Comparison Between Countries , 2018, Front. Psychol..

[8]  O. John,et al.  The Next Big Five Inventory (BFI-2): Developing and Assessing a Hierarchical Model With 15 Facets to Enhance Bandwidth, Fidelity, and Predictive Power , 2017, Journal of personality and social psychology.

[9]  Rebecca E. Martin,et al.  Promise and Paradox : Measuring Students ’ Non-Cognitive Skills and the Impact of Schooling , 2015 .

[10]  O. John,et al.  Anchoring Vignettes: Can They Make Adolescent Self-Reports of Social-Emotional Skills More Reliable, Discriminant, and Criterion-Valid? , 2016 .

[11]  Angela L. Duckworth,et al.  Measurement Matters , 2015, Educational researcher.

[12]  Lex Borghans,et al.  Fostering and Measuring Skills: Improving Cognitive and Non-Cognitive Skills to Promote Lifetime Success , 2014, SSRN Electronic Journal.

[13]  A. Poropat A meta-analysis of adult-rated child personality and academic performance in primary education. , 2014, The British journal of educational psychology.

[14]  J. Heckman,et al.  NBER WORKING PAPER SERIES UNDERSTANDING THE MECHANISMS THROUGH WHICH AN INFLUENTIAL EARLY CHILDHOOD PROGRAM BOOSTED ADULT OUTCOMES , 2012 .

[15]  J. Heckman,et al.  Achievement Tests and the Role of Character in American Life , 2013 .

[16]  Jonathan N. Wand Credible Comparisons Using Interpersonally Incomparable Data: Nonparametric Scales with Anchoring Vignettes , 2013 .

[17]  Rhonda N. T. Nese,et al.  Social Emotional Assets and Resilience Scales: Development of a strength-based short-form behavior rating scale system , 2012 .

[18]  J. Heckman,et al.  Hard Evidence on Soft Skills , 2012, Labour economics.

[19]  Deniz S Ones,et al.  An other perspective on personality: meta-analytic integration of observers' accuracy and predictive validity. , 2010, Psychological bulletin.

[20]  A. Poropat A meta-analysis of the five-factor model of personality and academic performance. , 2009, Psychological bulletin.

[21]  A. Caspi,et al.  The Power of Personality: The Comparative Validity of Personality Traits, Socioeconomic Status, and Cognitive Ability for Predicting Important Life Outcomes , 2007, Perspectives on psychological science : a journal of the Association for Psychological Science.

[22]  R. Robins,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES Personality Predictors of Academic Outcomes: Big Five Correlates of GPA and SAT Scores , 2007 .

[23]  J. Allik,et al.  Agreement Among Adolescents, Parents, and Teachers on Adolescent Personality , 2006, Assessment.

[24]  J. Raven,et al.  Manual for Raven's progressive matrices and vocabulary scales , 1962 .

[25]  G. Thomson,et al.  Methods of Estimating Mental Factors , 1938, Nature.

[26]  M. S. Bartlett,et al.  The statistical conception of mental factors. , 1937 .