Using qualitative methods to develop a survey measure of math and science engagement

Abstract Student engagement in math and science is vital to students' academic achievement and long-term participation in science, technology, engineering, and mathematic (STEM) courses and careers. In this study, we conducted in-depth interviews with 106 students from sixth to twelfth grade and 34 middle and high school teachers about how they conceptualized math and science engagement and disengagement. Our qualitative analysis of student and teacher interviews supported the multidimensional construct of engagement outlined in the academic literature. Our analysis also revealed additional indicators that have been included in prior measures of engagement less frequently. We then described how we used this qualitative information from students and teachers to develop and validate a new student self-report measure of math and science engagement.

[1]  J. Morse.  Qualitative data analysis (2nd ed): Mathew B. Miles and A. Michael Huberman. Thousand Oaks, CA: Sage Publications, 1994. Price: $65.00 hardback, $32.00 paperback. 238 pp , 1996 .

[2]  T. Goetz,et al.  Academic Emotions in Students' Self-Regulated Learning and Achievement: A Program of Qualitative and Quantitative Research , 2002 .

[3]  Jacqueline Ravet.  Making sense of disengagement in the primary classroom: a study of pupil, teacher and parent perceptions , 2007 .

[4]  G. Crombez,et al.  A Motivational Perspective on Coping with Pain , 2018, Oxford Scholarship Online.

[5]  Feifei Ye,et al.  The Math and Science Engagement Scales: Scale development, validation, and psychometric properties , 2016 .

[6]  Classroom Misbehavior in the Eyes of Students: A Qualitative Study , 2012, TheScientificWorldJournal.

[7]  A. Vibert,et al.  APPROACHES TO STUDENT ENGAGEMENT: DOES IDEOLOGY MATTER? , 2003 .

[8]  Sara E. Rimm-Kaufman,et al.  To what extent do teacher–student interaction quality and student gender contribute to fifth graders’ engagement in mathematics learning? , 2015 .

[9]  Linda C. Schmidt,et al.  Longitudinal relations of self-efficacy to outcome expectations, interests, and major choice goals in engineering students , 2008 .

[10]  Barbara A. Greene.  Measuring Cognitive Engagement With Self-Report Scales: Reflections From Over 20 Years of Research , 2015 .

[11]  E. Skinner,et al.  Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. , 1993 .

[12]  Toni Kempler Rogat,et al.  Affect and engagement during small group instruction , 2011 .

[13]  Kristin E. Voelkl.  Identification with School , 1997, American Journal of Education.

[14]  Ming-Te Wang,et al.  Staying Engaged: Knowledge and Research Needs in Student Engagement. , 2014, Child development perspectives.

[15]  Richard A. Krueger,et al.  Focus groups : a practical guide for applied research / by Richard A. Krueger , 1989 .

[16]  David A. Gillam,et al.  A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas , 2012 .

[17]  Benjamin C. Heddy,et al.  The Challenges of Defining and Measuring Student Engagement in Science , 2015 .

[18]  Denise Pope,et al.  Doing School: How We Are Creating a Generation of Stressed-Out, Materialistic, and Miseducated Students , 1999 .

[19]  B MilesMatthew,et al.  Qualitative Data Analysis , 2018, Approaches and Processes of Social Science Research.

[20]  Jacquelynne S. Eccles,et al.  Development During Early and Middle Adolescence. , 2006 .

[21]  Andrew J. Martin,et al.  Motivation and Engagement in English, Mathematics and Science High School Subjects: Towards an Understanding of Multidimensional Domain Specificity. , 2007 .

[22]  James J. Appleton,et al.  Student engagement with school: Critical conceptual and methodological issues of the construct. , 2008 .

[23]  Allan Wigfield,et al.  Reading motivation: A domain-specific approach to motivation , 1997 .

[24]  Andrew J. Martin,et al.  Exploring the Ups and Downs of Mathematics Engagement in the Middle Years of School , 2015 .

[25]  J. Connell,et al.  Relationships matter: linking teacher support to student engagement and achievement. , 2004, The Journal of school health.

[26]  Adam V. Maltese,et al.  Eyeballs in the Fridge: Sources of early interest in science , 2010 .

[27]  L. Harris,et al.  Secondary Teachers' Conceptions of Student Engagement: Engagement in Learning or in Schooling?. , 2011 .

[28]  Wendy McColskey,et al.  The Measurement of Student Engagement: A Comparative Analysis of Various Methods and Student Self-report Instruments , 2012 .

[29]  Linda L. Griffin,et al.  An Ecological Analysis of Middle School Misbehavior Through Student and Teacher Perspectives , 2003 .

[30]  J. Meece,et al.  Students' goal orientations and cognitive engagement in classroom activities , 1988 .

[31]  J. Finn.  Withdrawing From School , 1989 .

[32]  Katariina Salmela-Aro,et al.  The trajectories of student emotional engagement and school burnout with academic and psychological development: Findings from Finnish adolescents , 2015 .

[33]  Stuart A. Karabenick,et al.  Cognitive Processing of Self-Report Items in Educational Research: Do They Think What We Mean? , 2007 .

[34]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[35]  Lisa Linnenbrink-Garcia,et al.  Academic Emotions and Student Engagement , 2012 .

[36]  T. Kindermann,et al.  A Motivational Perspective on Engagement and Disaffection , 2009 .

[37]  Michael E. Woolley,et al.  Cognitive Pretesting and the Developmental Validity of Child Self-Report Instruments: Theory and Applications , 2004, Research on social work practice.

[38]  Charlotte Jonasson,et al.  Teachers and students' divergent perceptions of student engagement: recognition of school or workplace goals , 2012 .

[39]  D. Stull,et al.  Doing Team Ethnography: Warnings and Advice , 1997 .

[40]  Annemarie S. Palincsar,et al.  Motivating Project-Based Learning: Sustaining the Doing, Supporting the Learning , 1991 .

[41]  J. Reeve,et al.  Enhancing Students' Engagement by Increasing Teachers' Autonomy Support , 2004 .

[42]  J. Finn,et al.  Student engagement: What is it? Why does it matter? , 2012 .

[43]  David Zyngier,et al.  (Re)conceptualising Student Engagement: Doing Education Not Doing Time. , 2008 .

[44]  P. Pintrich,et al.  "Should I ask for help?" The role of motivation and attitudes in adolescents' help seeking in math class. , 1997 .

[45]  S. Christenson,et al.  Handbook of Research on Student Engagement , 2012 .

[46]  Janet Mancini Billson,et al.  Focus Groups: A Practical Guide for Applied Research , 1989 .

[47]  Hunter Gehlbach,et al.  Measure Twice, Cut down Error: A Process for Enhancing the Validity of Survey Scales , 2011 .

[48]  Angela M. O’Donnell and Cindy E. Hmelo-Silver.  Introduction What is Collaborative Learning?: An Overview , 2013 .

[49]  D. Rubio,et al.  Objectifying Content Validity: Conducting a Content Validity Study in Social Work Research , 2003 .

[50]  Catherine D. Ennis,et al.  Building Bridges to Student Engagement: Communicating Respect and Care for Students in Urban High Schools. , 2000 .

[51]  J. Reeve,et al.  Assessing students’ engagement: A review of instruments with psychometric Qualities , 2014 .

[52]  Doug Lombardi,et al.  Emotions in Science Education , 2014 .

[53]  Jennifer L. Glanville,et al.  The Measurement of School Engagement , 2007 .

[54]  Helen R. Quinn,et al.  A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas , 2013 .

[55]  Emma Little,et al.  Secondary school teachers’ perceptions of students’ problem behaviours , 2005 .

[56]  Jim Saxton,et al.  JOINT ECONOMIC COMMITTEE , 1998 .

[57]  David Zyngier,et al.  Listening to teachers–listening to students: substantive conversations about resistance, empowerment and engagement , 2007 .

[58]  Qiping Kong,et al.  Student engagement in mathematics: Development of instrument and validation of construct , 2003 .