A MULTILEVEL ANALYSIS OF THE EFFECT OF PROMPTING SELF-REGULATION IN TECHNOLOGY-DELIVERED INSTRUCTION

Two studies were conducted to examine the effect of prompting self-regulation, an intervention designed to improve learning from technology-delivered instruction. In Study 1, trainees who were prompted to self-regulate gradually improved their declarative and procedural knowledge over time, relative to the other conditions, whereas test scores declined over time for trainees who were not prompted to self-regulate. In Study 2, basic performance remained stable over time and strategic performance improved over time for trainees who were prompted to self-regulate, relative to the other conditions, whereas performance declined over time for trainees who were not prompted to self-regulate. Trainees’ cognitive ability moderated the effect of the prompts on basic performance and task-specific self-efficacy moderated the effect of the prompts on strategic performance. Prompting self-regulation resulted in stronger performance gains over time for trainees with higher ability or higher self-efficacy. These results demonstrate prompting self-regulation improved performance over time, relative to the other conditions, in both online, work-related training and laboratory settings. The results are consistent with theory suggesting self-regulation is a dynamic process that has a gradual effect on performance and highlight the importance of using a within-subjects design in self-regulation research.

[1]  A. Bandura.  Self-Efficacy: The Exercise of Control , 1997, Journal of Cognitive Psychotherapy.

[2]  Traci Sitzmann.  Self-Reported Learning: What Are We Really Measuring? , 2007 .

[3]  Gillian B. Yeo,et al.  Subjective cognitive effort: a model of states, traits, and time. , 2008, The Journal of applied psychology.

[4]  P. Ackerman,et al.  Motivation and cognitive abilities: an integrative/aptitude-treatment interaction approach to skill acquisition , 1989 .

[5]  C. Cooper,et al.  International review of industrial and organizational psychology , 1986 .

[6]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[7]  A. Bandura.  Social Foundations of Thought and Action , 1986 .

[8]  J. Colquitt,et al.  Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research. , 2000, The Journal of applied psychology.

[9]  Thomas C. Reeves,et al.  Pseudoscience in Computer-Based Instruction: The Case of Learner Control Research. , 1993 .

[10]  Eduardo Salas,et al.  Effects of Training Goals and Goal Orientation Traits on Multidimensional Training Outcomes and Performance Adaptability. , 2001, Organizational behavior and human decision processes.

[11]  Remus Ilies,et al.  Goal regulation across time: the effects of feedback and affect. , 2005, The Journal of applied psychology.

[12]  Richard E. Snow,et al.  Individual differences and the design of educational programs , 1986 .

[13]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[14]  M. David Merrill,et al.  Integrative goals for instructional design , 1990 .

[15]  Karen L. Murphy,et al.  Constructivism and collaboration on the Internet: case study of a graduate class experience , 1995 .

[16]  J. Cassady.  Self-Reported GPA and SAT: A Methodological Note , 2001 .

[17]  J. Hunter.  Cognitive ability, cognitive aptitudes, job knowledge, and job performance , 1986 .

[18]  Sheila Danko,et al.  Disentangling Achievement Orientation and Goal Setting: Effects on Self-Regulatory Processes , 2003 .

[19]  Clint A. Bowers,et al.  Networked simulations: New paradigms for team performance research , 1995 .

[20]  E. A. Locke,et al.  Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. , 2002, The American psychologist.

[21]  John E. Mathieu,et al.  Role of causal attributions in dynamic self-regulation and goal processes. , 1994 .

[22]  Philip H. Winne,et al.  A METACOGNITIVE VIEW OF INDIVIDUAL DIFFERENCES IN SELF-REGULATED LEARNING , 1996 .

[23]  C. Carver,et al.  Origins and Functions of Positive and Negative Affect: A Control-Process View. , 1990 .

[24]  Robert E. Ployhart,et al.  Growth Modeling Using Random Coefficient Models: Model Building, Testing, and Illustrations , 2002 .

[25]  J. Kevin Ford,et al.  LEARNING WITHIN A LEARNER CONTROL TRAINING ENVIRONMENT: THE INTERACTIVE EFFECTS OF GOAL ORIENTATION AND METACOGNITIVE INSTRUCTION ON LEARNING OUTCOMES , 2003 .

[26]  J. Singer,et al.  Applied Longitudinal Data Analysis , 2003 .

[27]  S. Kozlowski,et al.  Active learning: effects of core training design elements on self-regulatory processes, learning, and adaptability. , 2008, The Journal of applied psychology.

[28]  Eleanor M. Smith,et al.  Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer , 1998 .

[29]  Pierre-Jean Marescaux,et al.  Metacognition : process, function and use , 2002 .

[30]  Douglas Kauffman.  Self-Regulated Learning in Web-Based Environments: Instructional Tools Designed to Facilitate Cognitive Strategy Use, Metacognitive Processing, and Motivational Beliefs , 2004 .

[31]  K. Brown,et al.  Does self-regulation require cognitive resources? Evaluation of resource allocation models of goal setting. , 1996, The Journal of applied psychology.

[32]  A. Bandura,et al.  Self-evaluative and self-efficacy mechanisms governing the motivational effects of goal systems. , 1983 .

[33]  D. Broadbent,et al.  What makes interruptions disruptive? A study of length, similarity, and complexity , 1989 .

[34]  M. Ree,et al.  PREDICTING TRAINING SUCCESS: NOT MUCH MORE THAN g , 2006 .

[35]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[36]  F. Zijlstra,et al.  Temporal factors in mental work: Effects of interrupted activities , 1999 .

[37]  D. Detterman,et al.  Scholastic Assessment or g? , 2004, Psychological science.

[38]  Stephanie C Payne,et al.  A meta-analytic examination of the goal orientation nomological net. , 2007, The Journal of applied psychology.

[39]  F. Paas,et al.  Cognitive Architecture and Instructional Design , 1998 .

[40]  F. Schmidt,et al.  The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. , 1998 .

[41]  J. J. Donovan,et al.  Missing the mark: effects of time and causal attributions on goal revision in response to goal-performance discrepancies. , 2003, The Journal of applied psychology.

[42]  Christopher A. Monk,et al.  Recovering From Interruptions: Implications for Driver Distraction Research , 2004, Hum. Factors.

[43]  Jan de Leeuw,et al.  Introducing Multilevel Modeling , 1998 .

[44]  John Sweller,et al.  The impact of sequencing and prior knowledge on learning mathematics through spreadsheet applications , 2005 .

[45]  P. Ackerman,et al.  Goal setting, conditions of practice, and task performance: A resource allocation perspective. , 1994 .

[46]  William P. Eveland,et al.  User Control and Structural Isomorphism or Disorientation and Cognitive Load? , 2001, Commun. Res..

[47]  Malcolm James Ree,et al.  Intelligence Is the Best Predictor of Job Performance , 1992 .

[48]  Gillian B. Yeo,et al.  A multilevel analysis of effort, practice, and performance: effects; of ability, conscientiousness, and goal orientation. , 2004, The Journal of applied psychology.

[49]  E. Salas,et al.  Toward a science of distributed learning. , 2007 .

[50]  J. Bargh,et al.  The psychology of action : linking cognition and motivation to behavior , 1999 .

[51]  Jeffrey B Vancouver,et al.  Self-efficacy and resource allocation: support for a nonmonotonic, discontinuous model. , 2008, The Journal of applied psychology.

[52]  M. Knowles.  Self-directed learning , 1975 .

[53]  A. Bandura,et al.  Negative self-efficacy and goal effects revisited. , 2003, The Journal of applied psychology.

[54]  P. Karoly.  Mechanisms of Self-Regulation: A Systems View , 1993 .

[55]  N. Dorans.  CORRESPONDENCES BETWEEN ACT™ AND SAT® I SCORES , 1999 .

[56]  Ruth Kanfer,et al.  Self-regulatory and other non-ability determinants of skill acquisition. , 1996 .

[57]  T. S. Bateman,et al.  Self-regulation: from goal orientation to job performance. , 2006, The Journal of applied psychology.

[58]  E. E. Jones.  The Framing of Competence , 1989 .

[59]  S. Kozlowski,et al.  Adaptive Guidance: Enhancing Self-Regulation, Knowledge, and Performance in Technology-Based Training , 2002 .

[60]  J. Cellier,et al.  Interference between switched tasks , 1992 .

[61]  P. Ackerman,et al.  A self-regulatory skills perspective to reducing cognitive interference. , 1996 .

[62]  Robert P. Abelson,et al.  A Variance Explanation Paradox: When a Little is a Lot , 1985 .

[63]  Kent B. Monroe,et al.  Effect-Size Estimates: Issues and Problems in Interpretation , 1996 .

[64]  Siu L. Chow,et al.  Significance test or effect size , 1988 .

[65]  M. Frese,et al.  Self-regulation in error management training: emotion control and metacognition as mediators of performance effects. , 2005, The Journal of applied psychology.

[66]  S. Payne,et al.  The impact of error training and individual differences on training outcomes: an attribute-treatment interaction perspective. , 2002, The Journal of applied psychology.

[67]  S. Kozlowski,et al.  Goal orientation and ability: interactive effects on self-efficacy, performance, and knowledge. , 2002, The Journal of applied psychology.

[68]  Philip H. Winne,et al.  Inherent details in self-regulated learning , 1995 .

[69]  H. Akaike.  A new look at the statistical model identification , 1974 .

[70]  Stephanie B. Corliss,et al.  The effects of reflective prompts and collaborative learning in hypermedia problem-based learning environments on problem solving and metacognitive skills , 2005 .

[71]  Eduardo Salas,et al.  E-Learning in Organizations , 2005 .

[72]  Gilad Chen,et al.  Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. , 2000 .

[73]  Gilad Chen,et al.  Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. , 2000, The Journal of applied psychology.

[74]  Robert A. Wisher,et al.  THE COMPARATIVE EFFECTIVENESS OF WEB-BASED AND CLASSROOM INSTRUCTION: , 2006 .

[75]  Kenneth G. Brown,et al.  Using computers to deliver training: Which employees learn and why? , 2001 .

[76]  G. Latham,et al.  The Effect of Learning versus Outcome Goals on a Simple versus a Complex Task , 1996 .

[77]  Kurt Kraiger,et al.  A meta-analytic investigation of learner control: Old findings and new directions. , 2007 .

[78]  G E BRIGGS,et al.  The relative efficiency of several training methods as a function of transfer task complexity. , 1962, Journal of experimental psychology.

[79]  G E BRIGGS,et al.  Effects of task complexity and task organization on the relative efficiency of part and whole training methods. , 1963, Journal of experimental psychology.

[80]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[81]  Daniel G Bobrow,et al.  On data-limited and resource-limited processes , 1975, Cognitive Psychology.

[82]  Robert M. Yadrick,et al.  Alternating Task Modules in Isochronal Distributed Training of Complex Tasks , 1996, Hum. Factors.

[83]  Jeffrey B Vancouver,et al.  When self-efficacy negatively relates to motivation and performance in a learning context. , 2006, The Journal of applied psychology.

[84]  M. Frese,et al.  International Review of Industrial and Organizational Psychology , 2000 .

[85]  Deborah A. Prentice,et al.  When small effects are impressive , 1992 .