Co-Measure: developing an assessment for student collaboration in STEAM activities

BackgroundThe shortage of skilled workers choosing STEM (Science, Technology, Engineering, and Math) careers in the USA and worldwide has fueled a movement towards STEAM, in which the “A” addresses the arts and humanities. STEAM education has been proposed as a way to offer relevant problems to solve while drawing on creative and collaborative skills to increase interest and engagement in these fields. Despite the interest in increasing STEAM globally, research on the efficacy of instructional approaches, including ways to assess collaborative problem solving (CPS), is lacking.ResultsThis paper reports the development of a rubric, named Co-Measure, for researchers and educators to use to assess student collaboration, at the individual level, when students are working in K-12 STEAM activities. Our project team provides the rationale, process, validation, initial iterations to the rubric, and next steps to inform STEM researchers and move STEAM instruction and learning forward. A final rubric is provided and made freely available to researchers and educators.ConclusionsAs STEAM education gains popularity in K-12 schools, assessment of student collaboration is needed to identify the dimensions of the skill in order to provide appropriate problem solving opportunities within instruction. The assessment is also needed to adjust instruction when students are not offered opportunities to work collaboratively during STEAM activities. Utilizing existing generalized frameworks of CPS provided the initial guide to direct research specific to CPS in STEAM activities. Using an iterative process to identify and evaluate attributes of student behavior associated with CPS in STEAM classrooms by a project team comprised of learning scientists, educational researchers and psychometricians allowed for rigorous research while drawing on appropriate expertise. Co-Measure has the potential to be modified and broadly extended to assess CPS in elementary and post-secondary classrooms using STEAM instructional practices.

[1]  Nikol Rummel,et al.  Instructional Support for Collaboration in Desktop Videoconference Settings , 2005 .

[2]  L. Richard Meeth,et al.  Interdisciplinary Studies: A Matter of Definition , 1978 .

[3]  Brigid Barron,et al.  Problem Solving in Video-based Microworlds: Collaborative and Individual Outcomes of High-Achieving Sixth-Grade Students. , 2000 .

[4]  Xianglei Chen.  STEM Attrition: College Students' Paths into and out of STEM Fields. Statistical Analysis Report. NCES 2014-001. , 2013 .

[5]  J. Perry,et al.  Conceptualizing and Measuring Collaboration , 2007 .

[6]  Jeremy V. Ernst,et al.  Scientific and Technical Visualization in Technology Education. , 2007 .

[7]  Cassie Quigley,et al.  “Finding the Joy in the Unknown”: Implementation of STEAM Teaching Practices in Middle School Science and Math Classrooms , 2016 .

[8]  B. B. Morgan,et al.  An Analysis of Team Evolution and Maturation , 1993 .

[9]  C. MacCann,et al.  DEVELOPMENT AND VALIDITY EVIDENCE SUPPORTING A TEAMWORK AND COLLABORATION ASSESSMENT FOR HIGH SCHOOL STUDENTS , 2008 .

[10]  M. Shuttleworth.  Test Validity , 2019 .

[11]  Barbara Wasson,et al.  Designing for Change in Networked Learning Environments, Proceedings of the International Conference on Computer Support for Collaborative Learning, CSCL 2003, Bergen, Norway, June 14-18, 2003 , 2003, CSCL.

[12]  James W. Bequette,et al.  A Place for Art and Design Education in the STEM Conversation , 2012 .

[13]  Nikol Rummel,et al.  A rating scheme for assessing the quality of computer-supported collaboration processes , 2007, Int. J. Comput. Support. Collab. Learn..

[14]  Gregory K. W. K. Chung,et al.  Issues in the Computer-based Assessment of Collaborative Problem Solving , 2003 .

[15]  Michelle H. Land,et al.  Full STEAM Ahead: The Benefits of Integrating the Arts Into STEM , 2013, Complex Adaptive Systems.

[16]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[17]  Carolyn Penstein Rosé,et al.  Supporting CSCL with automatic corpus analysis technology , 2005, CSCL.

[18]  Gerald Knezek,et al.  STEAM for Student Engagement , 2013 .

[19]  J. Creswell.  Qualitative inquiry and research design: Choosing among five approaches, 2nd ed. , 2007 .

[20]  Patrick Griffin,et al.  Assessment and Teaching of 21st Century Skills , 2011 .

[21]  Robert D. Atkinson,et al.  Addressing the STEM Challenge by Expanding Specialty Math and Science High Schools. , 2007 .

[22]  Timothy Koschmann,et al.  Problematizing the Problem , 2003, CSCL.

[23]  Nadia Kellam,et al.  Steam as Social Practice: Cultivating Creativity in Transdisciplinary Spaces , 2014 .

[24]  Carolyn Penstein Rosé,et al.  Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning , 2008, Int. J. Comput. Support. Collab. Learn..

[25]  S. Jaramillo,et al.  Adams Becker, S.; Cummins, M.; Davis, A.; Freeman, A.; Hall Giesinger, C.; Ananthanarayanan, V. NMC Horizon Report: 2017 Higher Education Edition. Austin, Texas: The New Media Consortium, 2017 , 2017 .

[26]  Nikol Rummel,et al.  A new method to assess the quality of collaborative process in CSCL , 2005, CSCL.

[27]  Michelle K. Smith,et al.  Why Peer Discussion Improves Student Performance on In-Class Concept Questions , 2009, Science.

[28]  Friedrich W. Hesse,et al.  A Framework for Teachable Collaborative Problem Solving Skills , 2015 .

[29]  Robin S. Vande Zande.  Teaching Design Education for Cultural, Pedagogical, and Economic Aims , 2010 .

[30]  Michael Gettings.  Putting It All Together: STEAM, PBL, Scientific Method, and the Studio Habits of Mind , 2016 .

[31]  PISA 2015 collaborative problem ‐ solving framework , 2022 .

[32]  Jessica J. Andrews,et al.  Benefits, Costs, and Challenges of Collaboration for Learning and Memory , 2015 .

[33]  F. Wickson,et al.  Transdisciplinary research: characteristics, quandaries and quality , 2006 .

[34]  Patrick Griffin,et al.  The Changing Role of Education and Schools , 2012 .

[35]  Charles K. Fadel,et al.  21st Century Skills: Learning for Life in Our Times , 2009 .

[36]  Stephanie D. Teasley,et al.  The Construction of Shared Knowledge in Collaborative Problem Solving , 1995 .

[37]  M. A. Campion,et al.  The Knowledge, Skill, and Ability Requirements for Teamwork: Implications for Human Resource Management , 1994 .

[38]  Lei Liu,et al.  A Tough Nut to Crack: Measuring Collaborative Problem Solving , 2016 .

[39]  Kristina M. Whicker,et al.  Cooperative Learning in the Secondary Mathematics Classroom , 1997 .

[40]  G. Gweon,et al.  Towards effective group work assessment: even what you don’t see can bias you , 2017 .

[41]  Paul C. Bauer,et al.  Conceptualizing and Measuring , 2013 .