Process assessment in dynamic testing using electronic tangibles

Task solving processes and changes in these processes have long been expected to provide valuable information about children's performance in school. This article used electronic tangibles (concrete materials that can be physically manipulated) and a dynamic testing format (pretest, training, and posttest) to investigate children's task solving processes and changes in these processes as a result of training. We also evaluated the value of process information for the prediction of school results. Participants were N = 253 children with a mean age of 7.8 years. Half of them received a graduated prompts training; the other half received repeated practice only. Three process measures were used: grouping behaviour, verbalized strategies, and completion time. Different measures showed different effects of training, with verbalized strategies showing the largest difference on the posttest between trained and untrained children. Although process measures were related to performance on our dynamic task and to math and reading performance in school, the amount of help provided during training provided the most predictive value to school results. We concluded that children's task solving processes provide valuable information, but the interpretation requires more research.

[1]  R. Sternberg.  Beyond IQ: A Triarchic Theory of Human Intelligence , 1984 .

[2]  H A SIMON,et al.  HUMAN ACQUISITION OF CONCEPTS FOR SEQUENTIAL PATTERNS. , 1963, Psychological review.

[3]  G. Phye,et al.  Inductive Reasoning: A Training Approach , 2008 .

[4]  Janneke Verhaegh.  Effective use of fun with a tangible interaction console , 2007 .

[5]  Emile H. L. Aarts,et al.  In-game assessment and training of nonverbal cognitive skills using TagTiles , 2012, Personal and Ubiquitous Computing.

[6]  Ann L. Brown,et al.  Breakdowns in flexible use of information: Intelligence-related differences in transfer following equivalent learning performance , 1985 .

[7]  H. W. van Boxtel,et al.  Actualiteit en kwaliteit van normen : Een werkwijze voor het normeren van een leerlingvolgsysteem , 2015 .

[8]  Willem J. Heiser,et al.  Explanatory item response modeling of children's change on a dynamic test of analogical reasoning , 2013 .

[9]  W. Resing.  Dynamic Testing and Individualized Instruction: Helpful in Cognitive Education? , 2013, Journal of Cognitive Education and Psychology.

[10]  P. Stringer.  Dynamic assessment in educational settings: is potential ever realised? , 2018 .

[11]  Jens F. Beckmann,et al.  Dynamic assessment: a case of unfulfilled potential? , 2018 .

[12]  Carol S. Lidz,et al.  Leaning Toward a Consensus About Dynamic Assessment: Can We? Do We Want To? , 2014, Journal of Cognitive Education and Psychology.

[13]  Sara Price,et al.  The role of iPads in pre-school children's mark making development , 2015, Comput. Educ..

[14]  Samuel Greiff,et al.  Exploring the Relation between Time on Task and Ability in Complex Problem Solving , 2015 .

[15]  J. Holdnack,et al.  Interpreting Intelligence Test Results for Children with Disabilities: Is Global Intelligence Relevant? , 2007, Applied neuropsychology.

[16]  Julian Elliott,et al.  Progress in the inductive strategy-use of children from different ethnic backgrounds: a study employing dynamic testing , 2017 .

[17]  Wilma C. M. Resing,et al.  The Effect of Dynamic Testing With Electronic Prompts and Scaffolds on Children’s Inductive Reasoning: A Microgenetic Study , 2015, Journal of Cognitive Education and Psychology.

[18]  R. Sternberg,et al.  Recognizing, defining, and representing problems. , 2003 .

[19]  Hiroshi Ishii,et al.  Emerging frameworks for tangible user interfaces , 2000, IBM Syst. J..

[20]  Jettie Hoonhout,et al.  TagTiles: optimal challenge in educational electronics , 2007, TEI.

[21]  E. Hunt,et al.  Intelligence as an information-processing concept. , 1980, British journal of psychology.

[22]  Wilma C. M. Resing,et al.  Measuring Inductive Reasoning Skills: The Construction of a Learning Potential Test , 2020 .

[23]  Samuel Greiff,et al.  Complex Problem Solving in Educational Contexts—Something Beyond g: Concept, Assessment, Measurement Invariance, and Construct Validity , 2013 .

[24]  Robert S Siegler,et al.  Cognitive variability. , 2007, Developmental science.

[25]  Heiko Rölke,et al.  The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. , 2014 .

[26]  Caitlin Tenison,et al.  Detecting math problem solving strategies: An investigation into the use of retrospective self-reports, latency and fMRI data , 2014, Neuropsychologia.

[27]  Samuel Greiff,et al.  Understanding students' performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files , 2016, Comput. Hum. Behav..

[28]  Robert J. Sternberg,et al.  Dynamic Testing: The Nature and Measurement of Learning Potential , 2001 .

[29]  Chad W. Buckendahl,et al.  A Review of Strategies for Validating Computer-Automated Scoring , 2002 .

[30]  R. Siegler.  Emerging Minds: The Process of Change in Children's Thinking , 1996 .

[31]  Robert S. Siegler,et al.  Learning About Learning , 2004 .

[32]  J. Elliott,et al.  Dynamic testing with tangible electronics: measuring children's change in strategy use with a series completion task. , 2011, The British journal of educational psychology.

[33]  Samuel Greiff,et al.  Inductive reasoning, domain specific and complex problem solving: Relations and development , 2013 .

[34]  R. Fabio.  Dynamic Assessment of Intelligence Is a Better Reply to Adaptive Behavior and Cognitive Plasticity , 2005, The Journal of general psychology.

[35]  A. Kozulin.  Learning potential and cognitive modifiability , 2011 .

[36]  Emile H. L. Aarts,et al.  A development support bubble for children , 2011, J. Ambient Intell. Smart Environ..

[37]  M. Scheerer,et al.  Problem Solving , 1967, Nature.

[38]  J. Campione,et al.  Children's learning and transfer of inductive reasoning rules: studies of proximal development. , 1986, Child development.

[39]  Bor-Chen Kuo,et al.  Cognitive diagnostic models for tests with multiple-choice and constructed-response items , 2016 .

[40]  U. Neisser.  Members of the Task Force : , 2005 .

[41]  R. Siegler,et al.  What leads children to adopt new strategies? A microgenetic/cross-sectional study of class inclusion. , 2006, Child development.

[42]  Julian Elliott,et al.  A “picture” of children's potential for learning: Looking into strategy changes and working memory by dynamic testing , 2012 .

[43]  Yulia Dodonova,et al.  Faster on easy items, more accurate on difficult ones: Cognitive ability and performance on a task of varying difficulty , 2013 .

[44]  Lynn S. Fuchs,et al.  The Predictive Validity of Dynamic Assessment , 2008 .

[45]  W. Heiser,et al.  Working Memory and Dynamic Measures of Analogical Reasoning as Predictors of Children's Math and Reading Achievement , 2014 .

[46]  J. Campione.  Assisted Assessment , 1989, Journal of learning disabilities.

[47]  Martha W. Alibali,et al.  Learning new problem-solving strategies leads to changes in problem representation , 2009 .

[48]  Samuel Greiff,et al.  Computer-generated log-file analyses as a window into students' minds? A showcase study based on the PISA 2012 assessment of problem solving , 2015, Comput. Educ..

[49]  J. Elliott.  The Psychological Assessment of Children with Learning Difficulties , 2003 .

[50]  Patrick Griffin,et al.  Log-file data as indicators for problem-solving processes , 2017 .

[51]  R. Sternberg.  The concept of intelligence and its role in lifelong learning and success. , 1997 .

[52]  R. Weisberg.  Toward an integrated theory of insight in problem solving , 2015 .

[53]  Carol S. Lidz,et al.  Dynamic Assessment in Practice: Clinical and Educational Applications , 2006 .

[54]  M. Ashcraft,et al.  Telling stories: the perils and promise of using verbal reports to study math strategies. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[55]  K. Richardson,et al.  Does IQ Really Predict Job Performance? , 2015, Applied developmental science.

[56]  R. Sternberg,et al.  "Dynamic testing": Erratum. , 1998 .

[57]  W. Heiser,et al.  Dynamic Testing of Analogical Reasoning in 5- to 6-Year-Olds , 2016 .

[58]  Willem Fontijn,et al.  On the correlation between children's performances on electronic board tasks and nonverbal intelligence test measures , 2013, Comput. Educ..

[59]  Patrick Perret,et al.  Children’s Inductive Reasoning: Developmental and Educational Perspectives , 2015, Journal of Cognitive Education and Psychology.

[60]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[61]  R. Sternberg,et al.  Unities in inductive reasoning , 1983 .