Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity

Abstract This study examines the validity of the complex problem solving (CPS) test MicroDYN by investigating a) the relation between its dimensions – rule identification (exploration strategy), rule knowledge (acquired knowledge), rule application (control performance) – and working memory capacity (WMC), and b) whether CPS predicts school grades in different domains beyond WMC. A sample of n = 393 German high school students (age M = 17.07, SD = 1.12) completed the computer-based tests Memory Updating Numerical and the CPS scenario MicroDYN. Using structural equation modeling, WMC predicted rule knowledge and rule application, which remained substantially correlated after controlling for WMC. Rule knowledge predicted school grades in science and social studies beyond WMC, but not in language subjects. Explanations for the differential concurrent validity of CPS as well as prerequisites for valid CPS assessment are discussed.

[1]  R. L. Babcock Analysis of Adult Age Differences on the Raven's Advanced Progressive Matrices Test , 1994 .

[2]  T. Little,et al.  To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits , 2002 .

[3]  Heinz-Martin Süß,et al.  Intelligenz und komplexes Problemlösen , 1999 .

[4]  Samuel Greiff,et al.  Dynamic Problem Solving: A new measurement perspective , 2012 .

[5]  M. Buehner,et al.  Reasoning and Working Memory as Predictors of School Grades. , 2008 .

[6]  W. Wittmann,et al.  Investigating the paths between working memory, intelligence, knowledge, and complex problem solving performances via Brunswik-symmetry , 1999 .

[7]  D. Klahr,et al.  All other things being equal: acquisition and transfer of the control of variables strategy. , 1999, Child development.

[8]  B. Muthén,et al.  Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes , 1997 .

[9]  Cleotilde Gonzalez,et al.  The Relationships between Cognitive Ability and Dynamic Decision Making. , 2005 .

[10]  Lara M. Triona,et al.  Point and Click or Grab and Heft: Comparing the Influence of Physical and Virtual Instructional Materials on Elementary School Students' Ability to Design Experiments , 2003 .

[11]  Samuel Greiff,et al.  Dynamic Problem Solving , 2012 .

[12]  Ulman Lindenberger,et al.  Complex span versus updating tasks of working memory: the gap is not that deep. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[13]  Patrick C. Kyllonen,et al.  Reasoning ability is (little more than) working-memory capacity?! , 1990 .

[14]  Robert F. Lorch,et al.  Learning the control of variables strategy in higher and lower achieving classrooms: Contributions of explicit instruction and experimentation. , 2010 .

[15]  Detlef H. Rost,et al.  Akkuratesse selbstberichteter Zensuren , 2004 .

[16]  L. Thompson,et al.  The criterion validity of tasks of basic cognitive processes , 2006 .

[17]  Markus Bühner,et al.  Working memory, visual–spatial-intelligence and their relationship to problem-solving , 2008 .

[18]  O. Wilhelm,et al.  Working memory capacity - facets of a cognitive ability construct , 2000 .

[19]  J. Funke,et al.  Beyond IQ: A Latent State-Trait Analysis of General Intelligence, Dynamic Decision Making, and Implicit Learning , 2011 .

[20]  Detlev Leutner,et al.  The fuzzy relationship of intelligence and problem solving in computer simulations , 2002, Comput. Hum. Behav..

[21]  J. Funke Dynamic systems as tools for analysing human judgement , 2001 .

[22]  J. Raven,et al.  Manual for Raven's progressive matrices and vocabulary scales , 1962 .

[23]  M A Just,et al.  From the SelectedWorks of Marcel Adam Just 1990 What one intelligence test measures : A theoretical account of the processing in the Raven Progressive Matrices Test , 2016 .

[24]  Annette Kluge,et al.  Performance Assessments With Microworlds and Their Difficulty , 2008 .

[25]  M. Daneman,et al.  Working memory and language comprehension: A meta-analysis , 1996, Psychonomic bulletin & review.

[26]  Richard E. Mayer,et al.  Incorporating Problem Solving into Secondary School Curricula , 1997 .

[27]  Joachim Funke,et al.  Complex problem solving: a case for complex cognition? , 2010, Cognitive Processing.

[28]  S. Ian Robertson,et al.  Problem-solving , 2001, Human Thinking.

[29]  D. Klahr,et al.  Finding One’s Place in Transfer Space , 2011 .

[30]  Joachim Wirth,et al.  Computer-based Assessment of Problem Solving Competence , 2003 .

[31]  P. Carpenter,et al.  Individual differences in working memory and reading , 1980 .

[32]  Berndt Brehmer,et al.  Reliability and validity of performance measures in microworlds , 2002 .

[33]  André Beauducel,et al.  On the Performance of Maximum Likelihood Versus Means and Variance Adjusted Weighted Least Squares Estimation in CFA , 2006 .

[34]  Robert C. Calfee,et al.  Handbook of educational psychology , 2013 .

[35]  S. Embretson,et al.  Item response theory for psychologists , 2000 .

[36]  N. Sander Inhibitory and executive functions in cognitive psychology : an individual differences approach examining structure and overlap with working memory capacity and intelligence , 2005 .

[37]  Samuel Greiff,et al.  Complex problem solving : More than reasoning ? , 2017 .

[38]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[39]  Raymond J. Shaw,et al.  Effects of adult age on structural and operational capacities in working memory. , 1991, Psychology and aging.

[40]  Kristy J. Lauver,et al.  Do psychosocial and study skill factors predict college outcomes? A meta-analysis. , 2004, Psychological bulletin.

[41]  Stefan Krumm,et al.  Reasoning=working memory¿attention , 2005 .

[42]  L. Angeles Evaluating Cutoff Criteria of Model Fit Indices for Latent Variable Models with Binary and Continuous Outcomes , 2002 .

[43]  J. E. Tschirgi,et al.  Sensible reasoning: A hypothesis about hypotheses. , 1980 .

[44]  Jan L. Plass,et al.  Intelligence Assessment with Computer Simulations. , 2005 .

[45]  Samuel Greiff,et al.  The Process of Solving Complex Problems , 2012, J. Probl. Solving.

[46]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .