Exploring Multiple Goals Balancing in Complex Problem Solving Based on Log Data

Multiple goals balancing is an important but not yet fully validated dimension of complex problem solving (CPS). The present study used process data to explore how solvers clarify goals, set priorities, and balance conflicting goals. We extracted behavioral indicators of goal pursuit from the log data of 3,201 students on the third subtask of the “Ticket” task in the PISA 2012 CPS test. Cluster analysis was used to identify 10 groups that varied in goal pursuit behavior. Logistics and least-squares regression analysis were used to explore how goal pursuit affected task scores and CPS proficiency. The results showed that competent solvers clarified goals and weighed priorities more effectively. They also made trade-offs between conflicting goals. The importance of theoretically-driven log data analysis and coping strategies in the face of multiple goals conflict scenarios was discussed.

[1]  Samuel Greiff,et al.  A multitrait-multimethod study of assessment instruments for complex problem solving☆ , 2013 .

[2]  K. Holyoak,et al.  The Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure , 1996 .

[3]  Samuel Greiff,et al.  Understanding students' performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files , 2016, Comput. Hum. Behav..

[4]  Samuel Greiff,et al.  Complex problem solving : More than reasoning ? , 2017 .

[5]  Samuel Greiff,et al.  The Genetics Lab: Acceptance and psychometric characteristics of a computer-based microworld assessing complex problem solving , 2012 .

[6]  Joachim Funke,et al.  You Cannot Have Your Cake and Eat It, too: How Induced Goal Conflicts Affect Complex Problem Solving , 2010 .

[7]  Sharlene D. Newman,et al.  Complex Problem Solving , 2015 .

[8]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[9]  Samuel Greiff,et al.  Measuring complex problem solving , 2009 .

[10]  Andreas Fischer,et al.  Common Process Demands of Two Complex Dynamic Control Tasks: Transfer Is Mediated by Comprehensive Strategies , 2017, Front. Psychol..

[11]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[12]  Samuel Greiff,et al.  Computer-generated log-file analyses as a window into students' minds? A showcase study based on the PISA 2012 assessment of problem solving , 2015, Comput. Educ..

[13]  Peter J. Rousseeuw,et al.  Clustering by means of medoids , 1987 .

[14]  P. Frensch,et al.  Complex problem solving : the European perspective , 1995 .

[15]  Samuel Greiff,et al.  Assessment of Complex Problem Solving: What We Know and What We Don’t Know , 2016 .

[16]  Dietrich Dörner,et al.  Problemlösefähigkeit und Intelligenz. , 1983 .

[17]  Samuel Greiff,et al.  The Process of Solving Complex Problems , 2012, J. Probl. Solving.

[18]  Christian Hennig,et al.  Cluster-wise assessment of cluster stability , 2007, Comput. Stat. Data Anal..

[19]  Samuel Greiff,et al.  Measuring complex problem solving: an educational application of psychological theories , 2013 .

[20]  Andreas Fischer,et al.  Complex problem solving—single ability or complex phenomenon? , 2015, Front. Psychol..

[21]  Jan L. Plass,et al.  Intelligence Assessment with Computer Simulations. , 2005 .

[22]  Bruce D. Burns,et al.  The Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure , 1996, Cogn. Sci..

[23]  Samuel Greiff,et al.  Dynamic Problem Solving , 2012 .

[24]  D. Dörner,et al.  Complex Problem Solving: What It Is and What It Is Not , 2017, Front. Psychol..

[25]  Joachim Funke,et al.  Complex problem solving: a case for complex cognition? , 2010, Cognitive Processing.

[26]  J. Funke.  Dynamic systems as tools for analysing human judgement , 2001 .

[27]  Samuel Greiff,et al.  Exploring Exploration: Inquiries into Exploration Behavior in Complex Problem Solving Assessment , 2013, EDM.

[28]  S. Holden.  Education at a Glance , 2006, The SAGE Encyclopedia of Higher Education.

[29]  Joachim Funke,et al.  Dealing with dynamic systems: Research strategy, diagnostic approach and experimental results , 1992 .