Interdisciplinary research agenda in support of assessment of collaborative problem solving: lessons learned from developing a Collaborative Science Assessment Prototype

Abstract Evidence from labor-market economics and predictive validity studies in psychology suggests that collaborative problem solving (CPS) is an increasingly important skill for both academic and career success in the 21st century. While there is a general agreement that collaborative problem solving is an important skill, there is less agreement on how to build an assessment to measure it, especially at scale and as a standardized test. Developing the type of CPS assessment envisioned in this work will require interdisciplinary synergy, involving learning science, data science, psychometrics, and software engineering. In this conceptual paper, we present our identification and novel instantiation of five interdisciplinary research strands supporting the development of a CPS assessment. We discuss how these research strands can comprehensively address some of the shortcomings of existing CPS assessments, such as collecting and managing the data from the process of collaboration in structured log files, or considering a statistical definition of collaboration in the design of the collaborative tasks. We describe the Collaborative Science Assessment Prototype developed at Educational Testing Service (ETS) under the proposed interdisciplinary research agenda to illustrate how these research strands can be operationalized.