Measuring Students’ Social-Emotional Learning Among California’s CORE Districts: An IRT Modeling Approach

Measuring Students’ Social-Emotional Learning Among California’s CORE Districts: An IRT Modeling Approach With an increased appreciation of students’ social-emotional skills among researchers and policy makers, many states and school districts are moving toward a systematic process to measure Social-Emotional Learning (SEL). In this study, we examine the measurement properties of California's CORE Districts’ SEL survey administered to over 400,000 students in grades 3 to 12 during the 2015-16 school year. We conduct analyses through both classical test theory and item response theory frameworks, applying three different polytomous IRT models on both the full student sample and on separate samples from each grade. From these analyses, we summarize the psychometric properties of items at each grade level, compare items' functionality across grades, compare student outcomes from IRT models and the classical approach, make suggestions on approaches to modeling and scaling the SEL survey data, and identify items, by grade, that do not contribute positively to measurement of each outcome. Finally, we discuss policy implications in using SEL measures among educators, administrators, policy makers, and other stakeholders. Robert H. Meyer Education Analytics

[1]  V. Battistich,et al.  Effects of an Elementary School Intervention on Students' “Connectedness” to School and Social Adjustment During Middle School , 2003, Journal of Primary Prevention.

[2]  R. Brennan,et al.  Test equating : methods and practices , 1995 .

[3]  Allison B. Dymnicki,et al.  The impact of enhancing students' social and emotional learning: a meta-analysis of school-based universal interventions. , 2011, Child development.

[4]  J. Heckman Schools, Skills, and Synapses , 2008, Economic inquiry.

[5]  C. Segal MISBEHAVIOR, EDUCATION, AND LABOR MARKET OUTCOMES , 2013 .

[6]  Cornelis A.W. Glas,et al.  Modeling Country-Specific Differential Item Functioning , 2013 .

[7]  W. Cunningham,et al.  The Effects of Climate Change on a Small Open Economy , 2014 .

[8]  Angela L. Duckworth,et al.  Personality Psychology and Economics , 2011, SSRN Electronic Journal.

[9]  James J. Heckman,et al.  The myth of achievement tests : the GED and the role of character in American life , 2013 .

[10]  Nathan Mantel,et al.  Chi-square tests with one degree of freedom , 1963 .

[11]  Herbert W. Marsh,et al.  Negative item bias in ratings scales for preadolescent children: a cognitive-developmental phenomenon , 1986 .

[12]  Jeri Benson,et al.  THE IMPACT OF ITEM PHRASING ON THE VALIDITY OF ATTITUDE SCALES FOR ELEMENTARY SCHOOL CHILDREN , 1985 .

[13]  R. Abbott,et al.  Effects of social development intervention in childhood 15 years later. , 2008, Archives of pediatrics & adolescent medicine.

[14]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[15]  M. Greenberg,et al.  Early Social-Emotional Functioning and Public Health: The Relationship Between Kindergarten Social Competence and Future Wellness. , 2015, American journal of public health.

[16]  E. Muraki A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[17]  Joseph A Durlak,et al.  Promoting Positive Youth Development Through School-Based Social and Emotional Learning Interventions: A Meta-Analysis of Follow-Up Effects. , 2017, Child development.

[18]  Martin R. West,et al.  Development and implementation of student social-emotional surveys in the CORE Districts , 2017 .

[19]  G. Masters A rasch model for partial credit scoring , 1982 .

[20]  N. Dorans,et al.  CONSTRUCTED RESPONSE AND DIFFERENTIAL ITEM FUNCTIONING: A PRAGMATIC APPROACH1 , 1991 .

[21]  Hunter Gehlbach Seven Survey Sins , 2015 .

[22]  W. Haenszel,et al.  Statistical aspects of the analysis of data from retrospective studies of disease. , 1959, Journal of the National Cancer Institute.

[23]  C. Belfield,et al.  The Economic Value of Social and Emotional Learning , 2015 .

[24]  Gerty J. L. M. Lensvelt-Mulders,et al.  Taking Stock of Self-Control , 2011, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[25]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[26]  John N. Friedman,et al.  How Does Your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star , 2010, The quarterly journal of economics.

[27]  J. Bridgeland,et al.  The Missing Piece: A National Teacher Survey on How Social and Emotional Learning Can Empower Children and Transform Schools. A Report for CASEL. Executive Summary. , 2013 .

[28]  A. Bandura Self-Efficacy: The Exercise of Control , 1997, Journal of Cognitive Psychotherapy.

[29]  J. Heckman,et al.  A gradient of childhood self-control predicts health, wealth, and public safety , 2011, Proceedings of the National Academy of Sciences.

[30]  S. Reise,et al.  Using the Nominal Response Model to Evaluate Response Category Discrimination in the PROMIS Emotional Distress Item Pools , 2011 .

[31]  A. Poropat A meta-analysis of the five-factor model of personality and academic performance. , 2009, Psychological bulletin.

[32]  C. Sharp,et al.  An Item Response Theory Analysis of the Problem Gambling Severity Index , 2012, Assessment.

[33]  Neil J. Dorans,et al.  Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. , 1986 .

[34]  Leon Bieber,et al.  Mindset The New Psychology Of Success , 2016 .