Measuring the impact of technological scaffolding interventions on micro-level processes of self-regulated workplace learning

This paper reports on the findings of an exploratory study in which the effects of technological scaffolding interventions on micro-level processes of self-regulated learning in the workplace were investigated. Empirical research in the workplace has been much less represented than in formal education. Even less research is available that aimed to identify which technological scaffolding interventions, out of those available in a learning environment, had the highest influence on specific micro-level process of self-regulated learning. This paper reports on the findings of a case study conducted in the naturalistic settings of two organizations in Europe (N?=?53) for the period of two months. Trace data about the events of engagement with the technological scaffolding interventions and micro-level processes of self-regulated learning were collected. Both a transition graph based analysis of the temporal dependencies of the collected events and multiple linear regression analyses showed that an intervention that promoted social awareness had consistently the highest effect on all the micro-level processes used in the study. This intervention was followed by the intervention that offered system-generated recommendations about learning paths, learning activities and knowledge assets to stimulate engagement into the micro-level processes within the forethought or preparatory phase of self-regulated learning. These findings suggest that both the social and organizational contexts should be taken into account when developing interventions aimed at supporting the forethought and engagement phases. Further discussion about research, methodological, and learning technology design implications is provided. Effects of technological scaffolds on self-regulated workplace learning are studied.Protocol based on trace data is used for measurement in an exploratory study.Transition graphs and multiple linear regression are used for analysis.Scaffolds promoting social awareness had consistently the highest effect.System-generated recommendations of learning resources had also high effects.

[1]  Päivi Tynjälä,et al.  Toward a 3-P Model of Workplace Learning: a Literature Review , 2013 .

[2]  Philip H. Winne,et al.  Bootstrapping learner ’ s self-regulated learning , 2010 .

[3]  John Dunlosky,et al.  Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology , 2012 .

[4]  Mathieu Bastian,et al.  Gephi: An Open Source Software for Exploring and Manipulating Networks , 2009, ICWSM.

[5]  Dragan Gasevic,et al.  Leveraging Semantic Technologies for Harmonization of Individual and Organizational Learning , 2010, EC-TEL.

[6]  Allyson Hadwin,et al.  Innovative ways for using gStudy to orchestrate and research social aspects of self-regulated learning , 2010, Comput. Hum. Behav..

[7]  Bernhard Schmitz,et al.  A plea for self-regulated learning as a process: Modelling, measuring and intervening , 2011 .

[8]  Donatella Persico,et al.  Self-Regulated Learning in Technology Enhanced Learning Environments , 2011 .

[9]  Andrew McAfee,et al.  Enterprise 2.0: New Collaborative Tools for Your Organization's Toughest Challenges , 2009 .

[10]  Xi Zhang,et al.  Understanding the users' continuous adoption of 3D social virtual world in China: A comparative case study , 2014, Comput. Hum. Behav..

[11]  Patricia Ordóñez de Pablos,et al.  Culture effects on the knowledge sharing in multi-national virtual classes: A mixed method , 2014, Comput. Hum. Behav..

[12]  Philip H. Winne,et al.  How Software Technologies Can Improve Research on Learning and Bolster School Reform , 2006 .

[13]  B. Zimmerman,et al.  Self-regulated learning and academic achievement: Theory, research, and practice. , 1989 .

[14]  Steven J Durning,et al.  Exploring clinical reasoning in novices: a self-regulated learning microanalytic assessment approach , 2014, Medical education.

[15]  Daniel J. Brass,et al.  Network Analysis in the Social Sciences , 2009, Science.

[16]  Marek Hatala,et al.  Trace-based Micro-analytic Measurement of Self-Regulated Learning Processes , 2016, J. Learn. Anal..

[17]  B. Uzzi,et al.  The Sources and Consequences of Embeddedness for the Economic Performance of Organizations: The Network Effect , 1996 .

[18]  Catherine C. Chen,et al.  Self-Regulated Learning Strategies and Achievement in an Introduction to Information Systems Course , 2002 .

[19]  Xi Zhang,et al.  Effects of information technologies, department characteristics and individual roles on improving knowledge sharing visibility: a qualitative case study , 2012, Behav. Inf. Technol..

[20]  Andrea D. Ellinger.  Contextual factors influencing informal learning in a workplace setting: The case of “reinventing itself company” , 2005 .

[21]  Knud Illeris,et al.  Workplaces and Learning , 2018, Learning, Development and Education.

[22]  R. Azevedo,et al.  A Theoretical Review of Winne and Hadwin’s Model of Self-Regulated Learning: New Perspectives and Directions , 2007 .

[23]  Xi Zhang,et al.  Mapping development of social media research through different disciplines: Collaborative learning in management and computer science , 2015, Comput. Hum. Behav..

[24]  Shane Dawson,et al.  Identifying key actors for technology adoption in higher education: A social network approach , 2012 .

[25]  D. Ashton.  The Impact Of Organisational Structure and Practices On Learning in The Workplace , 2004 .

[26]  B. Zimmerman,et al.  Handbook of Self-Regulation of Learning and Performance , 2011 .

[27]  R. Azevedo.  Theoretical, conceptual, methodological, and instructional issues in research on metacognition and self-regulated learning: A discussion , 2009 .

[28]  Nada Dabbagh,et al.  Personal Learning Environments, social media, and self-regulated learning: A natural formula for connecting formal and informal learning , 2012, Internet High. Educ..

[29]  B. Zimmerman.  Theories of Self-Regulated Learning and Academic Achievement: An Overview and Analysis , 2001 .

[30]  Victoria J. Marsick,et al.  Toward a unifying framework to support informal learning theory, research and practice , 2009 .

[31]  Mor Naaman,et al.  Is it really about me?: message content in social awareness streams , 2010, CSCW '10.

[32]  Philip H. Winne,et al.  Exploring students’ calibration of self reports about study tactics and achievement , 2002 .

[33]  J. Pallant.  SPSS survival manual : a step-by-step guide to data analysis using SPSS version 15 , 2007 .

[34]  J. McCabe.  Metacognitive awareness of learning strategies in undergraduates , 2011, Memory & cognition.

[35]  Margaret Malloch,et al.  The Sage handbook of workplace learning , 2011 .

[36]  Marek Hatala,et al.  Associations between technological scaffolding and micro-level processes of self-regulated learning: A workplace study , 2016, Comput. Hum. Behav..

[37]  Miltiadis D. Lytras,et al.  Software Technologies in Knowledge Society , 2011, J. Univers. Comput. Sci..

[38]  Robert A. Bjork,et al.  The promise and perils of self-regulated study , 2007, Psychonomic bulletin & review.

[39]  M. Eraut,et al.  Informal learning in the workplace , 2004 .

[40]  Mingming Zhou.  Operationalizing and tracing goal orientation and learning strategy , 2008 .

[41]  Vive Kumar,et al.  Supporting Self-Regulated Learning with gStudy Software: The Learning Kit Project ** , 2006 .

[42]  R. Azevedo.  Using Hypermedia as a Metacognitive Tool for Enhancing Student Learning? The Role of Self-Regulated Learning , 2005 .

[43]  L. Lipponen,et al.  Models of Innovative Knowledge Communities and Three Metaphors of Learning , 2004 .

[44]  L. Freeman.  Centrality in social networks conceptual clarification , 1978 .

[45]  Daniel C. Moos,et al.  Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia? , 2008 .

[46]  P. Pintrich.  The role of goal orientation in self-regulated learning. , 2000 .

[47]  Xi Zhang,et al.  The Relationship between Incentives, Explicit and Tacit Knowledge Contribution in Online Engineering Education Project , 2012 .

[48]  Geraldine Clarebout,et al.  Metacognition and the use of tools , 2013 .

[49]  B. Tabachnick,et al.  Using multivariate statistics, 5th ed. , 2007 .

[50]  Konrad Tollmar,et al.  Supporting social awareness @ work design and experience , 1996, CSCW '96.

[51]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[52]  Patricia Ordóñez de Pablos,et al.  A Framework for Detecting and Removing Knowledge Overlaps in a Collaborative Environment , 2013, J. Web Eng..

[53]  Ying Ding,et al.  Applying centrality measures to impact analysis: A coauthorship network analysis , 2009, J. Assoc. Inf. Sci. Technol..

[54]  Allison Littlejohn,et al.  Charting Collective Knowledge: Supporting Self-Regulated Learning in the Workplace , 2009, 2009 Ninth IEEE International Conference on Advanced Learning Technologies.

[55]  Erik Duval,et al.  Context-Aware Recommender Systems for Learning: A Survey and Future Challenges , 2012, IEEE Transactions on Learning Technologies.

[56]  Xi Zhang,et al.  From e-learning to social-learning: Mapping development of studies on social media-supported knowledge management , 2015, Comput. Hum. Behav..

[57]  R. Bjork,et al.  Self-regulated learning: beliefs, techniques, and illusions. , 2013, Annual review of psychology.

[58]  Philip H. Winne,et al.  Improving Measurements of Self-Regulated Learning , 2010 .

[59]  V. Marsick,et al.  The Nature and Need for Informal Learning , 1999 .

[60]  L. Unwin,et al.  Workplace Learning : Main Themes & Perspectives , 2015 .

[61]  1 Theories of Work , Place and Learning : New Directions , 2022 .

[62]  Candice Burkett,et al.  Self-regulated Learning with MetaTutor: Advancing the Science of Learning with MetaCognitive Tools , 2010 .

[63]  Marek Hatala,et al.  Semantic Web and Linked Learning to Support Workplace Learning , 2012, LiLe@WWW.

[64]  Marek Hatala,et al.  Self-regulated Workplace Learning: A Pedagogical Framework and Semantic Web-based Environment , 2012, J. Educ. Technol. Soc..

[65]  Graham J Hole,et al.  How to Design and Report Experiments , 2002 .

[66]  Philip H. Winne,et al.  Learning Strategies, Study Skills, and Self-Regulated Learning in Postsecondary Education , 2013 .

[67]  Vive Kumar,et al.  Effects of self-regulated learning in programming , 2005, Fifth IEEE International Conference on Advanced Learning Technologies (ICALT'05).

[68]  M. Boekaerts,et al.  How Far Have We Moved Toward the Integration of Theory and Practice in Self-Regulation? , 2006 .

[69]  Mingming Zhou,et al.  Modeling Academic Achievement by Self-Reported versus Traced Goal Orientation. , 2012 .

[70]  Khanh-Phuong Thai,et al.  Supplemental Material for Habits and beliefs that guide self-regulated learning: Do they vary with mindset? , 2014, Journal of Applied Research in Memory and Cognition.

[71]  Fielding I. Winters,et al.  Self-Regulation of Learning within Computer-based Learning Environments: A Critical Analysis , 2008 .

[72]  Nada Dabbagh,et al.  Using Web-based Pedagogical Tools as Scaffolds for Self-regulated Learning , 2005 .

[73]  P. Tynjälä.  Perspectives into learning at the workplace , 2008 .

[74]  Allyson F. Hadwin,et al.  nStudy: Tracing and Supporting Self-Regulated Learning in the Internet , 2013 .

[75]  P. Reimann,et al.  Process mining techniques for analysing patterns and strategies in students’ self-regulated learning , 2013, Metacognition and Learning.

[76]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[77]  F. Dochy,et al.  Learning conditions for non‐formal and informal workplace learning , 2009 .

[78]  Roger Azevedo,et al.  A macro-level analysis of SRL processes and their relations to the acquisition of a sophisticated mental model of a complex system☆☆☆ , 2009 .

[79]  Donatella Persico,et al.  Detecting Self-Regulated Learning in Online Communities by Means of Interaction Analysis , 2008, IEEE Transactions on Learning Technologies.

[80]  Stevan E. Hobfoll,et al.  Communal Aspects of Self-Regulation , 2000 .

[81]  M. Puustinen,et al.  Models of Self-regulated Learning: A review , 2001 .

[82]  Philip H. Winne,et al.  Examining trace data to explore self-regulated learning , 2007 .

[83]  Allison Littlejohn,et al.  Self-regulated learning and knowledge sharing in the workplace: differences and similarities between experts and novices , 2009 .

[84]  B. Uzzi,et al.  Social Structure and Competition in Interfirm Networks: The Paradox of Embeddedness , 1997 .

[85]  Jeroen Janssen,et al.  Visualization of participation: Does it contribute to successful computer-supported collaborative learning? , 2007, Comput. Educ..

[86]  Allison Littlejohn,et al.  Self-regulated learning and knowledge sharing in the workplace , 2009 .

[87]  Tobias Ley,et al.  A methodology for eliciting, modelling, and evaluating expert knowledge for an adaptive work-integrated learning system , 2010, Int. J. Hum. Comput. Stud..

[88]  R. Azevedo,et al.  The Measurement of Learners’ Self-Regulated Cognitive and Metacognitive Processes While Using Computer-Based Learning Environments , 2010 .

[89]  Philip H. Winne,et al.  Measuring Self-Regulated Learning , 2000 .

[90]  K. Eisenhardt.  Building theories from case study research , 1989, STUDI ORGANIZZATIVI.

[91]  S. Järvelä,et al.  Sequential and temporal characteristics of self and socially regulated learning , 2014 .

[92]  Philip H. Winne,et al.  Issues in researching self-regulated learning as patterns of events , 2014 .

[93]  Andrea Landherr,et al.  A Critical Review of Centrality Measures in Social Networks , 2010, Bus. Inf. Syst. Eng..

[94]  Gregory Schraw,et al.  Measuring Self-Regulation in Computer-Based Learning Environments , 2010 .

[95]  S. Järvelä,et al.  Self-Regulated, Co-Regulated, and Socially Shared Regulation of Learning , 2011 .

[96]  Victoria J. Marsick,et al.  Researching workplace learning in the United States , 2011 .

[97]  Allison Littlejohn,et al.  Self-regulated learning behaviour in the finance industry , 2015 .

[98]  Yukiko Inoue,et al.  Online Education for Lifelong Learning , 2007 .

[99]  Katrien Verbert,et al.  Recommender Systems for Technology Enhanced Learning , 2014, Springer New York.

[100]  Xi Zhang,et al.  Effect of knowledge sharing visibility on incentive-based relationship in Electronic Knowledge Management Systems: An empirical investigation , 2013, Comput. Hum. Behav..

[101]  B. Zimmerman,et al.  Assessing Self-Regulation as a Cyclical, Context-Specific Phenomenon: Overview and Analysis of SRL Microanalytic Protocols , 2012 .

[102]  Jean Underwood,et al.  Self-regulated learning in technology enhanced learning environments: a European perspective , 2011 .

[103]  Barney Dalgarno,et al.  Implementing Web 2.0 technologies in higher education: A collective case study , 2012, Comput. Educ..

[104]  Xi Zhang,et al.  The impact of second life on team learning outcomes from the perspective of it capabilities , 2012 .

[105]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .