The impact of learner metacognition and goal orientation on problem-solving in a serious game environment

Abstract To understand the impact of learner metacognition and goal orientation on problem-solving, this study investigated 159 undergraduate students’ metacognition, goal orientations, and their problem-solving performances and processes in a laboratory setting using a Serious Game (SG) environment that adopts problem-based learning (PBL) pedagogy to teach space science. Utilizing cluster analysis, multiple regression, similarity measure and data visualization, this study analyzed multiple data sources, including computer log data, problem-solving performance scores, and survey data. The results show that both learner metacognition and goal orientation affected problem-solving. The findings of this study offer insights of how learner characteristics impact on problem-solving in SG environments with PBL pedagogy. It also contributes to understanding the design of SG environments to benefit learners based on their metacognitive levels.

[1]  Enjie Liu,et al.  Data Mining, Management and Visualization in Large Scientific Corpuses , 2016, Edutainment.

[2]  Janet E. Davidson,et al.  Smart Problem Solving: How Metacognition Helps , 1998 .

[3]  Roland Eils,et al.  circlize implements and enhances circular visualization in R , 2014, Bioinform..

[4]  Lucas Horton,et al.  Students' Problem Solving as Mediated by Their Cognitive Tool Use: A Study of Tool Use Patterns , 2007 .

[5]  J. Flavell.  Speculations about the nature and development of metacognition , 1987 .

[6]  Ole Smørdal,et al.  The Interaction between Design Features and Learners' Goals: A Case Study of a Science Museum Game , 2016, IxD&A.

[7]  Michael J. Middleton,et al.  Avoiding the demonstration of lack of ability: An underexplored aspect of goal theory. , 1997 .

[8]  Dirk Ifenthaler,et al.  Serious Games Analytics: Theoretical Framework , 2015 .

[9]  Jaejin Lee,et al.  Examining Through Visualization What Tools Learners Access as They Play a Serious Game for Middle School Science , 2015 .

[10]  Jonathan P. Rowe,et al.  Problem Solving and Game-Based Learning: Effects of Middle Grade Students' Hypothesis Testing Strategies on Learning Outcomes , 2011 .

[11]  Jason W. Osbourne,et al.  Four Assumptions of Multiple Regression That Researchers Should Always Test. , 2002 .

[12]  Jeffrey Earp,et al.  An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games , 2016, Comput. Educ..

[13]  John R. Mergendoller,et al.  Problem-Based Learning: Modifying the Medical School Model for Teaching High School Economics , 2001 .

[14]  Edgar Erdfelder,et al.  A short tutorial of GPower , 2007 .

[15]  Lu-Fang Lin.  Impacts of the Problem-based Learning Pedagogy on English Learners’ Reading Comprehension, Strategy Use, and Active Learning Attitudes , 2017 .

[16]  C. Wolters,et al.  Sense of Belonging and Self-Regulated Learning: Testing Achievement Goals as Mediators , 2018 .

[17]  Mark P. J. van der Loo,et al.  The stringdist Package for Approximate String Matching , 2014, R J..

[18]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[19]  J. Flavell.  Metacognition and Cognitive Monitoring: A New Area of Cognitive-Developmental Inquiry. , 1979 .

[20]  Reinhard Pekrun,et al.  A 3 × 2 achievement goal model , 2011 .

[21]  Jaime Sánchez,et al.  Problem solving and collaboration using mobile serious games , 2011, Comput. Educ..

[22]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[23]  Li Dengfeng,et al.  New similarity measures of intuitionistic fuzzy sets and application to pattern recognitions , 2002, Pattern Recognit. Lett..

[24]  Ming-Puu Chen,et al.  A computer game as a context for non-routine mathematical problem solving: The effects of type of question prompt and level of prior knowledge , 2009, Comput. Educ..

[25]  Patrick K. Freer.  Problem-Based Learning and Structural Redesign in a Choral Methods Course. , 2017 .

[26]  R. Mayer,et al.  Increased interestingness of extraneous details in a multimedia science presentation leads to decreased learning. , 2008, Journal of experimental psychology. Applied.

[27]  Christine Neuhold,et al.  Problem-Based Learning in European Studies , 2014 .

[28]  Min Liu,et al.  Creating a Multimedia Enhanced Problem-Based Learning Environment for Middle School Science: Voices from the Developers , 2014, Interdisciplinary Journal of Problem-Based Learning.

[29]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[30]  Huei-Tse Hou,et al.  Evaluating multiple aspects of a digital educational problem-solving-based adventure game , 2014, Comput. Hum. Behav..

[31]  Min Liu,et al.  Understanding the Connection between Cognitive Tool Use and Cognitive Processes as Used by Sixth Graders in a Problem-Based Hypermedia Learning Environment , 2004 .

[32]  Sandeep Chowdhry,et al.  Student’s perception of effectiveness of a technology enhanced problem based learning environment in a Mechanical Engineering module , 2016 .

[33]  Avi Kaplan,et al.  Motivational influences on transfer of problem-solving strategies , 2005 .

[34]  Min Liu,et al.  An analysis of cognitive tool use patterns in a hypermedia learning environment , 2005 .

[35]  C. Hmelo‐Silver.  Problem-Based Learning: What and How Do Students Learn? , 2004 .

[36]  Christian Sebastian Loh,et al.  Comparison of similarity measures to differentiate players' actions and decision-making profiles in serious games analytics , 2016, Comput. Hum. Behav..

[37]  J. Kruger,et al.  Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. , 1999, Journal of personality and social psychology.

[38]  Chung-Yuan Hsu,et al.  Visual behavior, flow and achievement in game-based learning , 2016, Comput. Educ..

[39]  A. Elliot,et al.  A 2 X 2 achievement goal framework. , 2001, Journal of personality and social psychology.

[40]  P. Pintrich.  A Manual for the Use of the Motivated Strategies for Learning Questionnaire (MSLQ). , 1991 .

[41]  John R. Savery,et al.  Overview of Problem-Based Learning: Definitions and Distinctions. , 2006 .

[42]  Johannes Strobel,et al.  When is PBL More Effective? A Meta-synthesis of Meta-analyses Comparing PBL to Conventional Classrooms , 2009 .

[43]  Min Liu,et al.  Thresholds of knowledge development in complex problem solving: a multiple-case study of advanced learners’ cognitive processes , 2013 .

[44]  Philippe Flajolet,et al.  Analytic Combinatorics of Chord Diagrams , 2000 .

[45]  A. Elliot,et al.  A HIERARCHICAL MODEL OF APPROACH AND AVOIDANCE ACHIEVEMENT MOTIVATION , 1997 .

[46]  R. Tamblyn,et al.  Problem-based learning : an approach to medical education , 1980 .

[47]  Diane L. Schallert,et al.  Middle School Focus: Examining the Interplay between Middle School Students Achievement Goals and Self-Efficacy in a Technology-Enhanced Learning Environment. , 2008 .

[48]  Annette F. Gourgey.  Metacognition in basic skills instruction , 1998 .

[49]  Eric M. Anderman,et al.  The Development and Validation of Scales Assessing Students' Achievement Goal Orientations , 1998, Contemporary educational psychology.

[50]  Aman Yadav,et al.  Problem‐based Learning: Influence on Students' Learning in an Electrical Engineering Course , 2011 .

[51]  B. Zimmerman.  From Cognitive Modeling to Self-Regulation: A Social Cognitive Career Path , 2013 .

[52]  B. Zimmerman.  Becoming a Self-Regulated Learner: An Overview , 2002 .

[53]  David H. Jonassen,et al.  Why Problem-Based Learning Works: Theoretical Foundations , 2014 .

[54]  Namsoo Shin,et al.  Predictors of well-structured and ill-structured problem solving in an astronomy simulation , 2003 .

[55]  Christian Sebastian Loh,et al.  Maximum Similarity Index (MSI): A metric to differentiate the performance of novices vs. multiple-experts in serious games , 2014, Comput. Hum. Behav..

[56]  Vinicius Cardoso Garcia,et al.  PBL in teaching computing: An overview of the last 15 years , 2013, 2013 IEEE Frontiers in Education Conference (FIE).

[57]  W. Schnotz,et al.  Accuracy of metacognitive judgments as a moderator of learner control effectiveness in problem-solving tasks , 2017 .

[58]  Fariha Gul,et al.  Relationship Between Metacognition, Goal Orientation and Academic Achievement , 2012 .

[59]  T. A. Ryan,et al.  Intentional behavior; an approach to human motivation , 1973 .

[60]  Min Liu.  The Effect of a Hypermedia Learning Environment on Middle School Students' Motivation, Attitude, and Science Knowledge , 2005 .

[61]  Gregory Schraw,et al.  Assessing metacognitive awareness , 1994 .

[62]  Fred Phillips,et al.  A Research Note on Accounting Students' Epistemological Beliefs, Study Strategies, and Unstructured Problem‐Solving Performance , 2001 .

[63]  Clark C. Abt,et al.  Serious games , 2016, Springer International Publishing.

[64]  Jaejin Lee,et al.  What We Can Learn from the Data: A Multiple-Case Study Examining Behavior Patterns by Students with Different Characteristics in Using a Serious Game , 2016, Technol. Knowl. Learn..

[65]  Roger Azevedo,et al.  Does adaptive scaffolding facilitate students ability to regulate their learning with hypermedia? q , 2004 .

[66]  E. A. Locke,et al.  Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. , 2002, The American psychologist.

[67]  D. Moshman.  Metacognitive Theories Revisited , 2018 .