The Potential of Complex Problem Solving Scenarios to Assess Students' Cognitive Abilities:

At the time when complex problem solving was established as a key aspect of today’s educational curricula and a central competence of international assessment frameworks like PISA, it became evident that the educational context places special demands on assessment instruments used for this purpose. In this chapter, we show how these challenges can successfully be addressed by reviewing recent advancements in the field of complex problem solving. We use the example of the Genetics Lab, a newly developed and psychometrically sound microworld, which emphasizes usability and acceptance amongst students, to discuss the challenges and opportunities of assessing complex problem-solving in the classroom.

[1]  A. Graesser,et al.  Domain-general problem solving skills and education in the 21st century , 2014 .

[2]  C. Dominik Güss,et al.  Cross-National Comparisons of Complex Problem-Solving Strategies in Two Microworlds , 2009, Cogn. Sci..

[3]  Mark D. Griffiths,et al.  The Structural Characteristics of Video Games: A Psycho-Structural Analysis , 2004, Cyberpsychology Behav. Soc. Netw..

[4]  D. Borsboom,et al.  The Theoretical Status of Latent Variables , 2003 .

[5]  J. Funke.  Dynamic systems as tools for analysing human judgement , 2001 .

[6]  Richard E. Mayer,et al.  Problem-solving transfer. , 1996 .

[7]  M. A. Anusuya,et al.  Human Intelligence , 1965, Nature.

[8]  Samuel Greiff,et al.  Complex problem solving : More than reasoning ? , 2017 .

[9]  Gayle S. Christensen,et al.  Where immigrant students succeed: A comparative review of performance and engagement in PISA 2003 , 2006 .

[10]  Stanley Rothman,et al.  Survey of Expert Opinion on Intelligence and Aptitude Testing. , 1987 .

[11]  P C Wason,et al.  Reasoning about a Rule , 1968, The Quarterly journal of experimental psychology.

[12]  Joachim Funke,et al.  Systematische Erforschung komplexer Problemlösefähigkeit anhand minimal komplexer Systeme , 2010 .

[13]  R. J. Repique.  Digital Natives, Digital Immigrants , 2013, Journal of the American Psychiatric Nurses Association.

[14]  Frank Jenkins,et al.  Assessing Complex Problem Solving Performances , 2003 .

[15]  A. Demetriou,et al.  Can we be intelligent about intelligence?: Why education needs the concept of plastic general ability , 2007 .

[16]  Sabine Krolak-Schwerdt,et al.  School placement decisions in Luxembourg: Do teachers meet the Education Ministry’s standards? , 2012 .

[17]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .

[18]  P. Frensch,et al.  Complex problem solving : the European perspective , 1995 .

[19]  Richard E. Mayer,et al.  Cognitive Theory of Multimedia Learning , 2021, The Cambridge Handbook of Multimedia Learning.

[20]  J. Funke.  Wissen über dynamische Systeme. Erwerb, Repräsentation und Anwendung , 1992 .

[21]  J. Quesada *,et al.  Complex problem-solving: a field in search of a definition? , 2005 .

[22]  Michael E. Martinez.  Future Bright: A Transforming Vision of Human Intelligence , 2013 .

[23]  Cleotilde Gonzalez,et al.  The Relationships between Cognitive Ability and Dynamic Decision Making. , 2005 .

[24]  Annette Kluge,et al.  Performance Assessments With Microworlds and Their Difficulty , 2008 .

[25]  A. Goldberger,et al.  Estimation of a Model with Multiple Indicators and Multiple Causes of a Single Latent Variable , 1975 .

[26]  U. Trautwein,et al.  The differential effects of school tracking on psychometric intelligence : Do academic-track schools make students smarter? , 2012 .

[27]  Andrew J. Martin,et al.  Problem solving and immigrant student mathematics and science achievement: Multination findings from the Programme for International Student Assessment (PISA) , 2012 .

[28]  Mary M. Omodei,et al.  The Fire Chief microworld generating program: An illustration of computer-simulated microworlds as an experimental paradigm for studying complex decision-making behavior , 1995 .

[29]  J. Gustafsson.  Schooling and intelligence: Effects of track of study on level and profile of cognitive abilities , 2001 .

[30]  Deanna Kuhn,et al.  Do Students Need to Be Taught How to Reason , 2009 .

[31]  Samuel Greiff,et al.  Sometimes less is more: Comparing the validity of complex problem solving measures ☆ , 2015 .

[32]  Denny Borsboom,et al.  Frontiers of Test Validity Theory: Measurement, Causation, and Meaning , 2013 .

[33]  J. Funke,et al.  Beyond IQ: A Latent State-Trait Analysis of General Intelligence, Dynamic Decision Making, and Implicit Learning , 2011 .

[34]  Anastasios A. Economides,et al.  The acceptance and use of computer based assessment , 2011, Comput. Educ..

[35]  Jim Ridgway,et al.  Using Computers to Assess New Educational Goals , 2003 .

[36]  Luis de Marcos,et al.  Gamifying learning experiences: Practical implications and outcomes , 2013, Comput. Educ..

[37]  Unité d'analyse,et al.  PISA 2012 FIELD TRIAL PROBLEM SOLVING FRAMEWORK SUBJECT TO POSSIBLE REVISION AFTER THE FIELD , 2012 .

[38]  H.L.J. van der Maas,et al.  A dynamical model of general intelligence: the positive manifold of intelligence by mutualism. , 2006, Psychological review.

[39]  K. Heller,et al.  Begabungsdiagnostik in der Schul- und Erziehungsberatung , 1991 .

[40]  Bruce D. Burns,et al.  The Impact of Goal Specificity on Strategy Use and the Acquisition of Problem Structure , 1996, Cogn. Sci..

[41]  S. Strohschneider,et al.  The Fate of the Moros: A Cross-cultural Exploration of Strategies in Complex and Dynamic Decision Making , 1999 .

[42]  Berndt Brehmer,et al.  Reliability and validity of performance measures in microworlds , 2002 .

[43]  Philip H. Winne,et al.  How Software Technologies Can Improve Research on Learning and Bolster School Reform , 2006 .

[44]  Jacob Cohen,et al.  The problem of units and the circumstance for POMP , 1999 .

[45]  Heinz-Martin Süß.  Intelligenz, Wissen und Problemlösen : Kognitive Voraussetzungen für erfolgreiches Handeln bei computersimulierten Problemen , 1996 .

[46]  Samuel Greiff,et al.  Cross-national gender differences in complex problem solving and their determinants , 2014 .

[47]  J. Funke.  Microworlds based on linear equation systems: A new approach to complex problem solving and experimental results , 1993 .

[48]  Joachim Funke,et al.  Complex problem solving: a case for complex cognition? , 2010, Cognitive Processing.

[49]  Vincent Koenig,et al.  Concept maps: A useful and usable tool for computer-based knowledge assessment? A literature review with a focus on usability , 2014, Comput. Educ..

[50]  D. Borsboom,et al.  The concept of validity. , 2004, Psychological review.

[51]  Rolf Loeber,et al.  Role of test motivation in intelligence testing , 2011, Proceedings of the National Academy of Sciences.

[52]  Bharath Sriraman,et al.  The International Handbook of Giftedness , 2009 .

[53]  Bengt Muthén,et al.  Latent variable modeling in heterogeneous populations , 1989 .

[54]  Michael E. Martinez.  Education As the Cultivation of Intelligence , 2000 .

[55]  J. Funke,et al.  Dynamisches Problemlösen: Entwicklung und Evaluation eines neuen Messverfahrens zum Steuern komplexer Systeme , 2005 .

[56]  R. Bagozzi,et al.  On the nature and direction of relationships between constructs and measures. , 2000, Psychological methods.

[57]  Samuel Greiff,et al.  A multitrait-multimethod study of assessment instruments for complex problem solving☆ , 2013 .

[58]  M. Prensky.  Digital Natives, Digital Immigrants Part 1 , 2001 .

[59]  Jason McPherson,et al.  Gs Invaders: Assessing a computer game-like test of processing speed , 2007, Behavior research methods.

[60]  U. Trautwein,et al.  Studie: "Herkunft zensiert?" : Leistungsdiagnostik und soziale Ungleichheiten in der Schule , 2013 .

[61]  Philip H. Winne,et al.  Examining trace data to explore self-regulated learning , 2007 .

[62]  Samuel Greiff,et al.  Complex Problem Solving in Educational Contexts—Something Beyond g: Concept, Assessment, Measurement Invariance, and Construct Validity , 2013 .

[63]  Samuel Greiff,et al.  The Process of Solving Complex Problems , 2012, J. Probl. Solving.

[64]  L. Gottfredson.  Mainstream science on intelligence: An editorial with 52 signatories, history, and bibliography , 1997 .

[65]  D. Washburn.  The games psychologists play (and the data they provide) , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[66]  C. Geiser,et al.  Stability and change in intelligence from age 12 to age 52: results from the Luxembourg MAGRIP study. , 2013, Developmental psychology.

[67]  Stephan Abele,et al.  Dynamische Problemlösekompetenz , 2012 .